Josef Klimke came from Silesia, a region then part of the Prussian empire with a substantial German population, but now mainly in southern Poland. His first language appears to have been either French or German. From about 1871 he was involved in mining, at first working as a miner, carpenter and bricklayer in coal and metal mines. He then studied geology, chemistry, mechanics, mine surveying and the theory of mining. For some time he attended geology classes at the University of Breslau (now Wroclaw, Poland), and in 1877 passed the German state examination in mine surveying and general mining. In 1880 he went to Venezuela where he worked on the gold-fields for three years. There he obtained (without further study) an engineering diploma from the University of Caracas. He eventually returned to Europe in 1888.
A syndicate based in Hamburg then sent him to investigate the gold-fields at Knysna, Cape Colony. Thereafter an English syndicate asked him to report on some properties on the Witwatersrand. He decided to remain in the South African Republic (Transvaal) to practice as a mining engineer. In September 1891 he was appointed as the first state mining engineer of the South African Republic (Transvaal) and settled in Pretoria. In this position he established the technical division of the Department of Mines and reported annually on its activities. These included the inspection of mines and of steam boilers, and gathering statistics on the use of mining machinery, the cost of mining, mining accidents, and so forth. In his report for 1894 he included information on the poisoning of cattle by cyanide solution discarded by the gold mines, the supply of electricity to the mines, and geological surveying of deep boreholes. His next report included a geological map of the Lydenburg district and information on coal deposits along the railway line to Delagoa Bay (now Baia de Maputo, 1895). Also in 1895 he imported large quantities of the ingredients to manufacture explosives. During 1896-1897 he visited Paris, France, to obtain equipment for a state laboratory. His last report was for the year 1897, by which time the first government geologist of the Republic, G.A.F. Molengraaff* had been appointed. In 1898 he published an account, in French, of the diamand deposits of the South African Republic in the Annales des Mines (Paris). He visited Paris again during 1898-1899 and while in Europe was awarded the Rothe Adler Orde, third class, by the German Kaiser and sent a chest of ore samples received as a present from the German government to the State Museum.
Klimke was elected an honorary member of the South African Association of Engineers and Architects in April 1894, and of the newly founded Chemical and Metallurgical Society of South Africa the same year. He served on the Board of Curators (Management Committee) of the State Museum (later the Transvaal Museum, now the Ditsong National Museum of Natural History) from its founding in 1892 to 1896 or 1897. In 1904 he applied for naturalisation as a British citizen of the Transvaal Colony.