I am indebted to Keith Warsop for the right to reproduce some of the following text from "The Early FA Cup Finals", a SoccerData publication of 2004.

The early players didn't need a referee! They were gentlemen, and any dispute could be settled by the two captains.  Of course, this idyllic state of affairs couldn't last, especially when competitive soccer in the form of the Football Association's Cup came along in 1872.

    The very first rules laid down for F.A. Cup games in 1871 specified that: ‘The Committee shall appoint two umpires and a referee to act at each of the matches in the Final Ties. Neither the umpires nor the referee shall be members of either of the contending Clubs and the decision of the umpires shall be final except in the case of the umpires disagreeing when an appeal shall be made to the referee, whose decision shall be final.’ These three officials were already a feature of football, especially in the most important games, though many ordinary club matches made do with just the two umpires, who were usually members of the contending clubs, a proceeding which was prohibited in the above F.A. Cup rule. Maurice Golesworthy has made the point that: ‘The wording of this rule shows just how much the increased competition was already beginning to change the attitude of the players. The days of the gentlemen players, who were proud of the manner in which rival teams generally conducted themselves, and settled any disputes, were beginning to fade. The time had come not only to specify the appointment of both umpires and referees but to insist upon these men being neutral.’

     The division of labour between the two umpires was that each of them looked after one half of the field. However, it is a mistake to suppose that their ‘half’ was that of the side they represented. In this context ‘one half of the field’ means dividing it by an imaginary line drawn between goal and goal; in other words the umpires operated like modern linesmen (now, of course, referees’ assistants) with the exception that they were inside the field of play rather than restricted to running the touchline. Also faulty is the supposition that, until the abolition of umpires in 1891, the referee stood on the touchline waiting to be called upon by them to adjudicate. In fact, there is pictorial evidence from the 1870s and 1880s which shows the referee inside the pitch and keeping up with the play.

     Shearman explains how the system worked: ‘Each side has its own umpire, who is armed with a stick or flag; the referee carries a whistle. When a claim for infringement of rules is made, if both umpires are agreed, each holds up his stick, and the referee calls the game to a halt by sounding his whistle. If one umpire allows the claim, and the referee agree with him, he calls a halt as before; if the other umpire and the referee agree that the claim be disallowed, the whistle is not sounded. Two of the three officials must therefore agree in allowing the claim or the whistle is silent, and players continue the game until the whistle calls them off. Both umpires and referee, therefore, must lose no time in arriving at a decision, or so much play is wasted.’

     Mention here of the referee’s whistle raises the question of when this was introduced. Modern references to its first use in 1878 in a match between Nottingham Forest and Sheffield Norfolk must clearly be wrong for the two clubs did not meet in that year. In fact their last encounter was on 17th January 1874 and reports of that game or earlier ones between the two clubs make no mention of a whistle though that is not necessarily proof that it was not used. However, the referee’s whistle and Forest have been linked  from at least 1891 when a booklet was issued in connection with a fund-raising bazaar in Nottingham in which it is stated: ‘The referee’s whistle was introduced by the Forest Club, and was used for the first time on the Forest Cricket Ground in the year 1878, when Sheffield Norfolk were trying conclusions with the Reds.’

     Therefore, one would expect to find some connection, even if not in 1878, and so it proves. In 1868 Walter Roe Lymbery became secretary-treasurer of Forest and his first account book survived into modern times in the possession of members of his family. In December 1872 he records spending fivepence ‘for an umpire’s whistle.’ No doubt in matches where a referee was also involved that official took over the whistle. So we can now take the use of a whistle by a football official back to December 1872, some six years before the generally accepted date. This is pretty well confirmed by R.M. Ruck’s 1928 article in which he recalls his time in the Royal Engineers team in the early 1870s: ‘It was considered the correct thing that, when the ball touched a player’s hand or arm, and “hand” was challenged, he at once threw up his hand and acknowledged it without waiting for the umpire’s whistle.’ Of course, it is probable that the use of the whistle took some time to spread throughout the game and at first, no doubt, the whistle was used to stop play, then the referee called time. Match reports into the early 1880s still use the phrase ‘time was called’ to note the end of the game.

     Incidentally, in February 1886 the Bangor Club wrote to the Football Association to ask ‘whether it is correct for umpires to have whistles’ and the F.A. Committee answered in the negative.

     Ruck’s mention of claiming ‘hands’ is a reminder of the fact that until 1891 everything had to be appealed for. As late as March 1891, just months before the abolition of umpires, the F.A. Council affirmed that an umpire had no right to give a decision until an appeal had been made by a player. Appealing quickly grew more prevalent during the 1870s and the sportsmanlike play which characterised the game in much of the pre-F.A. Cup era was soon diluted by the keeness of teams seeking to win the trophy. The Sportsman was severe on both finalists in 1875-76, those paragons of the sporting virtues, the Wanderers and the Old Etonians: ‘Several times “hands” were called by either side on the most frivolous pretexts but no advantage accrued therefrom.’ The replay brought further criticism: ‘Free kicks abounded through the same vexatious and unmeaning calls of “hands” which had been so frequent during the last match.’ Nevertheless, sportsmanship was not dead; in 1879-80 the same newspaper praised Clapham Rovers and Oxford University: ‘There was not one appeal in the game’, though there must have been one for Clapham’s goal.

    The referee was given greater powers in 1889/90, when he was allowed to award a free kick for foul play without waiting for an appeal. It was only at this point that I suspect he needed a whistle! The well known story that the referee's whistle was first used at a Nottingham Forest v. Sheffield Norfolk game in 1878 seens not to stand up to scrutiny! Forest didn't play Norfolk in 1878, and the referee had no need for a whistle anyway! Not until the 1891/92 season was he finally given the responsibilities he has to this day. The two umpires then assumed the role of linesmen, or "assistant referees" as we have called them from 1996 onwards.