Reverend Patrick H. Scully arrived at the Cape on 1 January 1820 to serve as the colony's first regular Roman Catholic clergyman since the British occupation in 1806. On 10 February he announced that a store in the Buitekant was to be opened as a church on Sunday 13 February. At first his duties were confined mainly to the soldiers, but in January 1821 the acting Governor, Sir Rufane Donkin, approved a modest salary for him as a civil clergyman. Later that year his community began to erect a place of worship, financed by subscriptions. However, his salary was stopped when the Governor, Lord Charles Somerset, returned from leave in December that year.
Towards the end of 1821 His Majesty's Astronomer at the Cape, Reverend Fearon Fallows*, who had set up a temporary observatory in Cape Town while a site for the buildings of the Royal Observatory, Cape of Good Hope, was being selected, appointed Scully as second assistant. He started work in January 1822. A few months later, in May 1822, he was promoted to first assistant, following the demotion of the previous incumbent, James Fayrer*, to labourer. Scully was well educated and soon quite capable of carrying out the observations and computations that his post required, as shown by Fallows's praise of his abilities in letters to John Barrow* at the British Admiralty, and to Sir John Herschel*, in May 1822: "I have every reason to speak well of him - no one can be more diligent or more desirous of doing his duty than he is" and "he is ...withal desirous of improving himself in the knowledge of the various parts of Practical Astronomy" (Warner, 1995, p. 78).
However, in July 1824 Scully was summarily dismissed by Fallows following "his inproprieties with a young house-maid... under my own roof" (Warner, 1979, p. 13). He left the colony on 11 July 1824 on the Venus, bound for London. Despite his indignation Fallows recommended to the Admiralty that Scully's salary be continued for six months to give him time to settle again in England, but this was refused. The date of his formal dismissal is recorded as 5 October 1824 (Gill, 1913).
Upon leaving the Cape Colony Scully entrusted his property, including it seems the Roman Catholic church, to two curators. In 1832 these two brought a case against Reverend Thomas Rishton to remove him from the premises of the church. Four years later a legal challenge by someone else sought to render invalid all claims to property of Patrick Scully.