Daphne Colless is a Kuku-Yalanji woman; her sister is Pauline Rice (q.v.). Colless went to school at Jervis Field in Ayr and then to Woree School after which she was encouraged to further her education in Townsville via a church scholarship; however, due to pressure from the non-Indigenous community, Colless' mother withdrew her before she was able to take the test. Instead Colless was sent to work in laundries and as a domestic.

Colless married at nineteen and had five children. After her youngest child was in school, Colless returned to the workforce with a position at the Meatworks in Qeerah. At the Meatworks, she became a union delegate but was sacked when she tried to change the system. Colless went to the Industrial Commission and was reinstated to her job. In 1973, she started work at the Department of Aboriginal and Islander Affairs as a liaison officer but only worked with them for eight months. She worked cleaning buses but she was sought by the Aboriginal Legal Service to work for them as a field officer, which she did for three and a half years.

In1974, Colless became a director of Douglas House, a rehabilitation home for Indigenous and non-Indigenous people with alcoholic problems.

In 1977, Colless won a position with the State Government Aboriginal Commission and was given a small desk in the Department of Aboriginal and Islander Advancement. Here she faced a complete lack of support and cooperation. The Department would provide no transport to access communities, no response to requests for information, no secretarial support and no telephone. Other departments, for example the Department of Health, were not permitted to release information to her. Colless was fired from this position for publicly criticising the State Government.

In 1978, Colless became Manager of Douglas House after the previous, non-Indigenous, manager was fired for financial mismanagement. While there, Colless established a community farm at the Queensland Tablelands and a meal service for the homeless in the parks.