The swearing-in took place at the Senate on Wednesday.
Mr Gumau emerged winner of the Bauchi South senatorial election held on August 11. He was declared winner by the returning officer of the election, Ahmed Fadam, who supervised the final collation of the results.
Mr. Gumau, who was a member of the House of Representatives, polled a total of 119,489 votes beating his closest rival, Ladan Salihu of the PDP who polled 50,256 votes. He replaces a late senator, Ali Wakili, who died on March 17.
Mr Babba-Kaita was declared winner of the Katsina North senatorial election by the Independent National Electoral Commission (INEC), scoring 224,607 votes, while his elder brother and a retired customs officer, Kabir Babba-Kaita of the PDP scored 59,724 votes.
He replaces a late senator, Mustapha Bukar, who died on April 4.