The main purpose of this study is to: * Learn about the safety of ubamatamab and to find out what dose of ubamatamab can be given alone or with cemiplimab to patients with ovarian cancer or cancer of the uterus * The study will also look at the levels of ubamatamab and/or cemiplimab in the body and measure how well the body can remove the study drug(s). This is called pharmacokinetics * The study will also look at any signs that ubamatamab alone or with cemiplimab can treat recurrent advanced ovarian cancer or cancer of the uterus * To find out how safe and tolerable pretreatment is in combination with ubamatamab and to see how well it works to prevent or minimize Cytokine Release Syndrome (CRS)