The "/robots.txt" page:

Return text/plain which is the content of the "robots-txt" setting, if such a setting exists and is non-empty. Or construct an RFC-9309 complaint robots.txt file and return that if there is not "robots.txt" setting.

This is useful for robot exclusion in cases where Fossil is run as a stand-alone server in its own domain. For the more common case where Fossil is run as a CGI, or SCGI, or a server that responding to a reverse proxy, the returns robots.txt file will not be at the top level of the domain, and so it will be pointless.