Brian joined the club as a full-back, where he had played for Merton Secondary and Hinchley Wood Country Schools, in the juniors in 1954 and worked his way up through the A team and reserves, converting to inside-right along the way, before making his first team début in the 1957/58 season. Strong, fast and with an eye for goal he was soon coveted by professional club, including an approach from Leyton Orient, soon signing amateur forms for Chelsea.

Timed at 10.4 seconds for the 100 yards and an all-round cricketer, he was an integral part of the side that won the Isthmian League three times in successive seasons and the Amateur Cup, playing for the England amateur XI and representing Great Britain in the 1964 Olympics qualifying game against Iceland, when he unfortunately suffered a broken leg. He recovered and stayed through the conversion to the professional ranks, returning from a second broken leg to his original full-back position and remaining at the club until the 1968/69 season, working for a firm of estate agents and surveyors.