The following year, 1906 was reported to have been one of the blackest in the history of the Association; when membership slumped to 28 and its finances were in a parlous state. In the early part of the year it was necessary to reduce the salary of the Secretary from £15 to £10 a month and, later, to inform him that, the position would have to be honorary. The Secretary, W. Walker, who had been appointed in March 1903, at the same time as W.R. Poynton had been elected President, could not continue under those conditions and tendered his resignation. He was succeeded by James T. Brown who acted as Secretary and Honorary Treasurer until 1909 when he moved to Johannesburg to become the Secretary of the National Federation. During 1906 the wages of all artisans were reduced and, in an effort to retain its few remaining members, the Association's annual subscription was reduced from five guineas to three. (£5-5-0 to £3-3-0)There was no improvement in trading conditions during 1907 when the third Annual Congress of the National Federation was held in Durban. From a business and social point of view, this Congress was a great success but the Association did not have sufficient funds to meet the expenses of £85. An appeal was made to twelve members asking them each to contribute £3 10s.
At that Congress, J.Z. Drake from Cape Town, acting as President in place of F. Turner of Pretoria, who had resigned during his term of office spoke of the depression gripping the country. Two years before, he said, the weekly wage bill of the industry had been £100,000, but that figure had been reduced to £20,000. His opinion was that the Industry would remain in the doldrums for five years unless it were to be given unexpected stimulus. He was to be proved correct.
At the end of 1908, membership increased to 38 but, of these, only 30 were in good standing. The upswing did not begin until 1910 when the wages of artisans were increased by a shilling a day, thus restoring the cut which had been made four years earlier. The Federation Congress that year was held in Johannesburg and, to defray the expenses of the Durban delegates, donations were collected from Merchants and others connected with the industry. Prosperity gradually returned to South Africa and, by the end of 1911, the Association was to clear all liabilities and had a credit balance of £55.
The following two years saw the improvement maintained and artisans benefited from two wage increases at the same time there was a small, but steady rise in membership. In January 1913 the Association took over the Board Room in Anglo-African House for its meetings and this venue was used for over twenty one years. In August 1914, the momentum which the Industry had so painfully regained was once again slowed down by the outbreak of the first World War.
After James Brown had resigned from the position of Secretary and Treasurer to take up his duties in Johannesburg, Ivor H. Geen was appointed to the post. He continued until February 1911 when his illness made it necessary for T. Hughes to become Acting Secretary with Thomas Midgley as Honorary Treasurer.
When Geen died during May, E. Laverack became the sixth Secretary to serve the Association during its first ten year existence. Laverack occupied the position until 1917 and was succeeded by J. Owen Jones who subsequently found that his own business enterprise did not allow him sufficient time to carry out this function. On 10 January 1918, B.E. Swinbourne, Chartered Accountant, was appointed Secretary, a position he was to hold until his retirement on 31 December 1951.
Bert (Swinny) Swinbourn served as a trooper in the South African Anglo Boer War and after the hostilities ceased returned to England. After qualifying as an Accountant he returned to Durban to start his own practice. He was honoured by the Association and the National Federation on many occasions for his great contribution and outstanding service to the Building Industry in South Africa over a period spanning more than 30 years.
The year 1918 ended with 51 members.