A user agent is a type of software application that retrieves and renders web content so that users can interact with it. The most common examples of user agents are web browsers and email readers.
A user agent acts as the intermediary between a web server and an online user. It sends a characteristic string in its HTTP request header that identifies the user agent to the server. Using this information, servers can determine the user agent’s operating system, application type, or software version. Common user agents include browsers such as Mozilla Firefox, Google Chrome, and Safari.
Other applications can also function as user agents, including crawlers (like Googlebot), game consoles, screen readers, link checkers, and legacy operating systems. By editing a robots.txt file, you can issue specific instructions to a search engine user agent, such as telling Googlebot which pages to crawl or avoid.