Joseph Downton and Lydia Williams, married in 1837 but James was probably their illegitimate son.

Joseph and Lydia had other children and ended up at Landshire Farm, West Quantoxhead, Somerset, where they stayed until the 1870s. At the baptisms of his sons, Joseph was described as a Yeoman. St Audries was the dedication of the church at West Quantoxhead.

In his 70s Joseph left the farm and kept The Devonshire Arms in Long Sutton.

He and Lydia then decamped to Croydon, Surrey, where Lydia died in 1884 and Joseph in 1891. Their son Joseph S Dowton was living with wife and child in Croydon, so this is likely to be where they died.