From “The Civil War Dictionary by Mark M. Boatner III page 17
The terms arsenal, armory, depot, and laboratory need definition. An arsenal is "a place of deposit for ordnance and ordnance stores," says Scott's Military Dictionary of 1861, "but there are also arsenals of construction and repairs." An armory, likewise, can be a place of manufacture or deposit, according to the same source. Subsequent military dictionaries, Wilhelm ( 1881), Garber and Bond (1936), bear out the synonymity of the words arsenal and armory. The official army dictionary of today (S.R. 320-5-1, 1950) says that an arsenal has the primary mission of manufacture but may also store and issue. The reader can therefore expect to find these terms used in various senses. An arsenal in the strictest sense was a depository which had under its supervision an armory (which did the manufacturing of arms), a laboratory ·(which made ammunition as well as setting standards and construction procedures), and a depot (which was located at some detached place and could have the missions of collection, repair, and issue of arms). "Some depots became arsenals and vice versa," say Albaugh and Simmons in their Confederate Arms, p. 76, which should be consulted for organization of the Confederate ordnance operations.
As far as Ordnance Bureau, the North had an “Ordnance Department” under the War Department while the South had an “Ordnance Bureau” (or actually a “Bureau of Ordnance “). For the CSA, the War Department was established by the Confederate Provisional Congress on February 26, 1861. It in turn had several sub-departments, including the “Bureau of Ordnance” established April 8, 1861.