Bank of America said there was no evidence of criminal activity
Several members of the US Senate are among those affected, who could now be vulnerable to identity theft.
Senate sources say the missing tapes may have been stolen from a plane by baggage handlers.
The bank gave no details of how the records disappeared, but said they had probably not been misused.
Customers' accounts were being monitoring and account holders would be notified if any "unusual activity" was detected, bank officials said.
Investigation
Bank of America said the tapes went missing in December while being shipped to a back-up data centre.
"We, with federal law authorities, have done a very robust, thorough investigation on this and neither we nor they would make the statement lightly that we believe those tapes to be lost," Alexandra Tower, a spokeswoman for the North Carolina-based bank, told Time magazine.
But although there was no evidence of criminal activity, the bank said, the Secret Service - a federal agency whose brief includes investigations of serious financial crime - is said to be looking into the loss.
New York Senator Charles Schumer said he was told by the Senate Rules Committee that the tapes were probably stolen from a commercial plane.
"Whether it is identity theft, terrorism, or other theft, in this new complicated world baggage handlers should have background checks and more care should be taken for who is hired for these increasingly sensitive positions," the Democratic senator said.
Details of his Vermont colleague Pat Leahy's credit card account are among those missing, Senator Leahy's spokeswoman Tracy Schmaler said.
About 900,000 military and civilian staff at the defence department are among the 1.2 million affected, according to a Pentagon spokesman.
One of the biggest domestic banks in the United States, Bank of America, has admitted to losing computer tapes containing 1.2 million federal employee accounts,  including the accounts of several U.S. senators, in a statement by the bank. According to the Pentagon, most of the accounts belong to  staff and civilians in the Department of Defense. The bank said the tapes were lost in December 2004 as they were being transported to a  data back-up centre by a commercial plane.

Currently, the U.S. Secret Service are looking in to the matter, a federal agency whose brief includes investigations of serious financial crime such as this. All parties concerned are worrying about possible identity theft as it contained valuable information such as bank account numbers, names and addresses.