This project is an AI agent that can control a web browser through natural language conversation, demonstrating its capability by sending an email through the Gmail web interface.

CRITICAL REQUIREMENT NOTE: This solution does not use any APIs (like Gmail API or SMTP) for the core task. It uses Playwright for genuine browser automation to open a real browser instance, navigate to gmail.com, and interact with the UI elements just as a human would.

(Here you would add a GIF or video of the agent working)

The project is intentionally split into small, focused modules so that conversation logic, content generation, and browser automation can evolve independently.

flowchart LR
    U[User] --> A[app.py\nStreamlit chat UI]
    A --> B[agent.py\nReasoningAgent]
    B --> C[llm_handler.py\nGemini content generation]
    B --> D[browser_controller.py\nPlaywright automation]
    D --> E[(Gmail Web UI)]
    D --> S[(screenshots/)]
    D --> T[(browser_state/state.json)]
    C --> A
    D --> A

• app.py is the orchestration layer. It owns the Streamlit interface, stores chat state, and routes the conversation into either content generation or browser automation.
• agent.py is the decision layer. It keeps track of the information gathered so far, asks Gemini what to do next, and turns that decision into a user-facing response.
• llm_handler.py is the content layer. It uses Gemini to draft a subject and body for the email in structured JSON so the rest of the system can consume it reliably.
• browser_controller.py is the execution layer. It launches Playwright, signs into Gmail, composes the email, sends it, captures screenshots, and persists browser state for later runs.

1. The user sends a request in Streamlit.
2. ReasoningAgent inspects the conversation state and decides whether it needs more information, should generate email content, or should start browser automation.
3. If content is needed, Gemini drafts the subject and body in llm_handler.py.
4. If browser actions are needed, browser_controller.py opens Gmail, reuses saved session state when available, and performs the login and send flow.
5. Screenshots and status messages are streamed back into the chat so the user can follow each step.

• browser_state/state.json stores Playwright storage state so repeat runs can skip login when the session is still valid.
• screenshots/ stores visual checkpoints from each major browser action, which makes the automation easy to inspect and debug.

The agent provides screenshots directly within the chat interface after each browser action. This is achieved by having the BrowserController yield the path to a newly taken screenshot, which the Streamlit app then immediately renders.

Automating logins on modern platforms like Gmail is challenging due to security measures.

Solution: This agent uses Playwright's persistent browser context.

1. On the first successful login, the agent saves the session's authentication state (cookies, local storage) to a browser_state/state.json file.
2. On subsequent runs, it loads this state. This allows the browser to be "already logged in," effectively and reliably bypassing the need for repeated 2FA or CAPTCHA challenges.
3. If the state is invalid or it's the first run, the browser runs in headless=False mode, allowing the user to manually solve a CAPTCHA if necessary.

• Frontend: Streamlit
• Browser Automation: Playwright
• AI/LLM: Google Gemini Pro
• Language: Python

1. Clone the repository:

   git clone <your-repo-url>
   cd gmail-browser-agent

2. Set your Google Gemini API Key: You must set your API key as an environment variable. You can get a free key from Google AI Studio.

   For macOS/Linux:

   export GOOGLE_API_KEY="your_key_here"

   For Windows (Command Prompt):

   set GOOGLE_API_KEY="your_key_here"

   For Windows (PowerShell):

   $env:GOOGLE_API_KEY="your_key_here"

3. Install dependencies:

   pip install -r requirements.txt
   playwright install

4. Run the Streamlit application:

   streamlit run app.py