Authenticated scraping guide

Collect from logged-in sites through persistent profiles.

Login-walled data needs a session, and sessions need continuity. Persistent profiles hold the login while your collector works at a pace a real user could sustain.

Guide updated 2026-09-01. Follow the steps in order.

Direct answer

Log into the target once in an AliasMode profile, then start and connect over CDP for each collection run via the Local API, keeping one account per site per profile and gentle rate limits.

What you will accomplish

Before you start

  • Accounts on the target sites that your terms of service permit you to use this way
  • AliasMode installed and the Local API available at 127.0.0.1:50400
  • A Playwright or Puppeteer collector
  • A residential proxy per profile matching the account's usual region

Step-by-step

  1. Confirm you may collect, then how much

    Check the site's terms, robots directives, and your account agreement. Decide a sustainable rate — pages per minute and runs per day — before writing code.

  2. One profile per site account

    Create an AliasMode profile per target account: scrape·site·account-name. If a site bans an account, the blast radius is one profile.

  3. Match the proxy to the account

    Attach a residential proxy in the region the account normally logs in from, and preflight. A familiar IP is the cheapest anti-challenge measure there is.

  4. Seed the login by hand

    Log in manually once inside the profile and solve any challenge personally. Collectors never handle the login form; they inherit the session.

  5. Start the session per run

    Start the profile through the Local API at the beginning of each run.

    const res = await fetch('http://127.0.0.1:50400/api/v1/browser/start?user_id=' + profileId);
    const { data } = await res.json();
    const browser = await chromium.connectOverCDP(data.ws.puppeteer);
  6. Collect at a human pace

    Throttle requests, randomize pacing modestly, and stop on rate-limit signals rather than pushing through. Long, slow, boring runs beat fast, hot ones.

    const context = browser.contexts()[0];
    for (const url of queue) {
      await page.goto(url);
      // extract, then wait — keep it slow and steady
      await page.waitForTimeout(3000 + Math.random() * 4000);
    }
  7. Stop and sync

    Close the browser and stop the profile after each run so the session is saved cleanly for tomorrow.

    await browser.close();
    await fetch('http://127.0.0.1:50400/api/v1/browser/stop?user_id=' + profileId);
  8. Monitor session health

    If a run lands on a login page or challenge, stop that profile, re-seed the login manually, and investigate the rate before resuming.

The AliasMode workflow

  1. Profiles as site credentials

    One profile per site account keeps logins, cookies, and fingerprint history attributable to one identity.

  2. Local API as the run switch

    Start and stop sessions from your collector's scheduler; nothing runs outside a managed session.

  3. Preflight per run

    Confirm the exit IP region before each run so the account never sees a surprise geo.

Verify it worked

  • Runs land on logged-in pages with no login flow executed by code.
  • Collection rates match the documented throttle, sampled from run logs.
  • Stopping a run mid-way leaves a stoppable, resumable state — no orphan browser.
  • Account health is stable: no new challenges or warnings across a month of runs.

Cautions

  • Scraping can breach a site's terms even when the data itself is public — get the legal basis right for your jurisdiction and data types.
  • Never collect personal data without a lawful basis; authenticated access is not consent.
  • Pushing through rate limits risks the account and, on shared IPs, the proxy's reputation.

The collection run loop

  1. Pre-run

    Preflight proxy, confirm rate limits, check yesterday's run ended clean.

  2. Start

    Local API start returns the CDP endpoint; connect your collector.

  3. Collect

    Throttled extraction with stop-on-challenge guards.

  4. Post-run

    Stop the profile, store the data, log session health.

Collection guardrails

Guardrails that keep accounts alive
GuardrailSettingWhy
Request pacingSeconds between pages, randomizedMachine-gun pacing is the top account killer
Daily volumeCap pages per account per dayVolume spikes read as abuse
Stop conditionsHalt on login pages and challengesPushing through turns a warning into a ban
Run windowsCollect at plausible hours4 a.m. marathons are a pattern

Authenticated scraping FAQ

Is authenticated scraping legal?

It depends on terms, data types, and jurisdiction. Get a legal review for your targets; the tooling here assumes you have the right to collect what you collect.

Why not log in inside the script?

Login forms carry the highest challenge risk. Log in once by hand in the profile, and your code only reuses a stable session.

How many accounts per site should I run?

Prefer one healthy account per site per project. Extra accounts multiply your maintenance and risk for little throughput.

Sources and verification

  • IETF · Robots Exclusion Protocol (RFC 9309) (checked September 2026)
  • Playwright · Browser.connectOverCDP API reference (checked September 2026)
  • Chrome DevTools Protocol · DevTools Protocol documentation (checked September 2026)

Public product details can change after the check date. Facts are re-checked on a monthly cycle.