The robots.txt file of Target site provides mixed messages for web scraping purposes. The directives within the file include both Allow and Disallow rules for different user-agents, including wildcard (), suggesting that the website's developers considered different types of bots during development. For instance, Disallow: /co-cart and Disallow: /checkout directives forbid all user-agents from accessing and scraping these specific paths. However, Allow: / directive is also stated right before the aforementioned disallow rules, which typically implies all user agents are allowed to crawl the website's general pages, the common focus of web scraping. This notwithstanding, the presence of substantial Disallow commands for various specific paths signals intended restrictions for most web scrapers. Careful interpretation of these rules is necessary to ensure compliance with the site's crawling directives while performing web scraping tasks.

These terms make it clear that Target.com and its associated services must not be accessed or used in an unauthorized or automated manner. The terms specifically mention, 'you must not access or use the site...through any automated, unethical or unconventional means' which alludes to web scraping and data collection techniques used by bots or crawlers.

In the case of APIs, it is not explicitly stated in the terms, but it can be inferred that usage would come under the same restricted conditions. Violations could lead to 'civil and criminal penalties including possible monetary damages', which Target.com alludes to in their course of action against such events. This indicates that proper permission and ethical consideration need to be given utmost importance while accessing or collecting data from the site.