The robots.txt file of Zillow has a tailored set of rules which lay down the conditions for web scraping on their website. For all user-agents other than recognized ones like the Googlebot or Bingbot, most of the directives state Disallow: /, limiting broad access to the site's URLs. Important sections like /homes/ and /homedetails/ are thus disallowed, along with others like /howto/* and /homedetail/ReportHome*, which might normally be the primary focus of a web scraping initiative. Effectively, this allows Zillow to limit data gathering activities by unrecognised scrapers.

The robots.txt file, however, does allow scraping under specific conditions. This involves the use of a wildcard (*) related to specific path segments, such as Allow: /homedetails/*zpid for all the user-agents, thus restricting access to pages possessing a specific pattern in their URLs. For instance, an endpoint with a valid zpid after /homedetails/ can be scraped. Thus, the scraping of specific data remains possible if the requests adhere strictly to the directives and pattern requirements as mentioned in the robots.txt. Overall, this file illustrates Zillow's intent to restrict broad unauthorized scraping while still allowing targeted data gathering under specific conditions.

Zillow's terms and conditions explicitly disallow automated data collection including web scraping. They emphasize this in multiple sections, stating 'you agree not to use or provide software (except for general purpose web browsers and email clients, or software expressly licensed by us) or services that interact or interoperate with ZG Services, e.g., for downloading, uploading, posting, flagging, emailing, search, or mobile use'. In this statement, Zillow implies that any scraping activity or use of scraping tools is not allowed unless expressly licensed by them.

Additionally, Zillow highlights the ramifications for violations of these prohibitions, which include termination of the agreement, immediate cessation of use of their services, and the potential for legal action. They note 'we may take any technical and legal steps to prevent the violation of this provision and to enforce these Terms', which underscores the seriousness with which Zillow treats the protection of their site's data from automated collection mechanisms.