How to Track Your Google Search Rankings Daily

May 28, 2026 · 11 min read

Anyone running SEO for a brand, agency client, or a portfolio of sites eventually needs the same artifact: a daily scoreboard that shows the exact position of a target URL for every tracked keyword, in every country that matters, with day-over-day deltas. Search Console will not give you that — it reports aggregates and lags by days. Ahrefs and SEMrush will, but at a per-keyword cost that punishes wide keyword lists and offers limited control over country and device. The build-your-own version is roughly 50 lines of Python plus a SERP-scrape endpoint, and it gives you the one number your client cares about in the Monday report: "what position am I in today, and how did that change from yesterday?"

This guide walks the full pipeline: defining the tracked-keyword list, scraping the SERP once per day per country, computing the position of the target URL in the organic block, persisting day-over-day diffs, and surfacing the result in a CSV your account manager can drop straight into a slide.

Why a Custom Rank Tracker Beats the Off-the-Shelf Options

The honest field comparison: Search Console reports average position aggregated across queries Google chose to surface, lagged 1–3 days. Useful for impression-volume context, useless as a Monday-morning scoreboard. The Custom Search JSON API runs on a Programmable Search Engine subset of the index — the positions it returns regularly disagree with the real SERP by 10+ slots, which is why no serious rank tracker uses it. Ahrefs and SEMrush both work, but their per-keyword billing makes it economically painful to track 5,000+ long-tail keywords for a content site, and neither lets you cleanly pin a country-and-device combination outside the presets they expose.

A custom SERP-scrape tracker fixes all three: you decide the keyword list, the country, the device, the schedule, and what gets stored. The cost shape is per-request rather than per-keyword-per-month, which inverts the unit economics for wide long-tail tracking.

The other quiet advantage is data ownership. A SaaS rank tracker keeps your historical SERP data in their warehouse. A self-built tracker writes a parquet file per day to your storage, which means you can run any analysis you want — keyword cannibalization, featured-snippet ownership over time, organic-CTR estimation by position — without paying for an export tier.

The third advantage is transparency over methodology. Every commercial tracker reports a position number without telling you what country, device, language, and personalization profile generated it. When a client asks "why does your tracker say I'm position 4 but I just searched and I'm position 6?", the SaaS answer is "we use a different data source" — which is true and unhelpful. A self-built tracker is the SERP your client just looked at because you control every input. That auditability matters when a paying client is questioning whether a rankings report justifies last month's retainer.

What the SERP Endpoint Returns

Per query, a /search/google/search job returns the rendered SERP parsed into four blocks:

Block	What's inside
organic	The ordered list of organic results — position, title, url, displayed_url, description, sitelinks
ads	Sponsored results above and below the organic block — labeled ad_position, kept separate from organic
paa	People Also Ask box — the expandable Q&A panel Google injects mid-SERP
related	Related searches at the bottom of the page

For rank tracking, only organic matters. The position field is the ordinal position in the organic block (1-indexed), which is what every rank-tracking product reports as "rank". Sponsored slots are explicitly excluded so that an ad above your organic result does not artificially push your reported position down — which is exactly what every other commercial tracker does.

The other fields you care about per organic result are url (the destination), displayed_url (what Google shows as the green URL — sometimes a different subdomain or path), and sitelinks (the indented secondary links Google attaches to dominant results, which is itself a ranking signal worth logging).

Defining the Tracked-Keyword List

This is the part most rank-tracker rollouts get wrong. The instinct is to track every keyword in the keyword research file. The right approach is to tier the list into three bands:

• Money keywords — 20 to 100 head and mid-tail terms with direct commercial intent. Track these daily, every country that matters, both desktop and mobile if device matters.
• Tracking-set keywords — 500 to 2,000 long-tail terms that represent the content portfolio's footprint. Track weekly, single country, single device.
• Discovery keywords — everything else from Search Console queries with > 10 impressions. Track monthly to detect new pages ranking.

The pipeline below handles all three bands with the same endpoint — only the schedule differs. A simple CSV defines the tracked set:

keyword,country,language,target_domain,band
best CRM for small business,us,en,example.com,money
crm software comparison,us,en,example.com,money
free crm trial,gb,en,example.com,money
hubspot vs salesforce,us,en,example.com,tracking
zoho crm review,us,en,example.com,tracking

The target_domain column lets the same pipeline track different domains for different keywords (useful for agencies running this across multiple clients) without splitting the input file.

One detail that catches every new rank-tracker rollout: the country code matters more than people expect. The same query — say, "best CRM software" — returns materially different SERPs in us, gb, ca, and au, because Google biases toward local results, regional brands, and country-specific subdomains. For a US SaaS targeting US buyers, tracking the gb SERP for the same keyword is not "redundant tracking" — it is tracking a different ranking environment. A US-targeted site that ranks position 3 in us and nowhere in gb is leaving an entire market on the table, and the rank tracker is the artifact that surfaces that asymmetry.

The API Call

Every LogPose Google Search endpoint is asynchronous — submit a job, poll until done, fetch the result. Submit with curl first to confirm the parameters are correct:

curl -G "https://api.logposervices.com/api/v1/search/google/search" \
  -H "X-API-Key: lp_xxxxxxx" \
  --data-urlencode "q=best CRM for small business" \
  --data-urlencode "pages=2" \
  --data-urlencode "country=us" \
  --data-urlencode "language=en"
# → {"job_id": "gs_8f3a..."}

curl -H "X-API-Key: lp_xxxxxxx" \
  "https://api.logposervices.com/api/v1/jobs/gs_8f3a?wait=true&timeout=60"

curl -H "X-API-Key: lp_xxxxxxx" \
  https://api.logposervices.com/api/v1/jobs/gs_8f3a/result

Each page returns roughly 10 organic results, so pages=2 gives the top 20 — which is the standard depth for rank tracking. Going deeper than position 20 has diminishing analytical value: positions 21+ rarely receive measurable organic traffic, and the noise floor (page-to-page result variance from index freshness) dominates the signal.

The four filters that matter for rank tracking are country, language, time_range, and in_title. The first two pin the SERP to the geography you actually care about. time_range=day is useful for news-sensitive verticals where the SERP changes daily. in_title is a debugging tool for checking which of your pages Google has indexed for a specific title token.

For competitive analysis the sites and exclude_sites filters become useful. Setting sites=example.com,competitor.com returns a SERP restricted to those two domains, which is the fastest way to answer "how many pages do I have indexed for this query vs my competitor" without parsing the full SERP. exclude_sites=reddit.com,quora.com is the inverse — strip the user-generated noise out of a head term to see the editorial SERP underneath, which is the actual SERP your content competes in.

The Python Pipeline

This is the script that runs on a cron. It reads the keyword CSV, scrapes one SERP per row, computes the position of the target domain in the organic block, and writes a rankings.csv with day-over-day deltas.

import os, time, csv, json
from datetime import date, timedelta
from urllib.parse import urlparse
import requests

API_KEY = os.environ["LOGPOSE_API_KEY"]
BASE = "https://api.logposervices.com/api/v1"
HEADERS = {"X-API-Key": API_KEY}


def submit_and_wait(path: str, params: dict, timeout_s: int = 90) -> dict:
    r = requests.get(f"{BASE}/{path}", params=params, headers=HEADERS, timeout=30)
    r.raise_for_status()
    job_id = r.json()["job_id"]
    deadline = time.time() + timeout_s
    while time.time() < deadline:
        s = requests.get(f"{BASE}/jobs/{job_id}", headers=HEADERS, timeout=15).json()
        if s["status"] == "completed":
            break
        if s["status"] == "failed":
            raise RuntimeError(s.get("error", "unknown failure"))
        time.sleep(2)
    else:
        raise TimeoutError(f"job {job_id} did not finish in {timeout_s}s")
    return requests.get(f"{BASE}/jobs/{job_id}/result", headers=HEADERS, timeout=15).json()


def domain_of(url: str) -> str:
    host = urlparse(url).netloc.lower()
    return host[4:] if host.startswith("www.") else host


def position_in_serp(serp: dict, target_domain: str) -> int | None:
    """Return the 1-indexed position of the first organic result whose
    domain matches target_domain, or None if not in the top 20."""
    target = target_domain.lower().lstrip(".")
    for i, row in enumerate(serp.get("organic", []), start=1):
        if domain_of(row.get("url", "")) == target:
            return i
    return None


def track_one(keyword: str, country: str, language: str, target: str) -> dict:
    serp = submit_and_wait(
        "search/google/search",
        {"q": keyword, "pages": 2, "country": country, "language": language},
    )
    return {
        "keyword": keyword,
        "country": country,
        "target": target,
        "position": position_in_serp(serp, target),
        "top_url": (serp.get("organic") or [{}])[0].get("url"),
        "serp": serp,  # keep for later analysis
    }

The position_in_serp function is the entire rank-tracking algorithm: walk the organic block, return the first index where the result's domain matches the target. Subdomain handling — should blog.example.com count as a hit for example.com? — is a policy choice. The version above does exact-host match; the agency-friendly version is the apex-domain match below.

def apex(host: str) -> str:
    """Best-effort apex domain — drop the first label if there are 3+ labels."""
    parts = host.split(".")
    if len(parts) <= 2:
        return host
    # Handle common 2-label TLDs (co.uk, com.au, etc.) by keeping last 3 labels
    if parts[-2] in {"co", "com", "org", "net", "gov", "ac"}:
        return ".".join(parts[-3:])
    return ".".join(parts[-2:])

Swap domain_of(row.get("url", "")) == target for apex(domain_of(...)) == apex(target) to count subdomain matches as hits — which is what most clients actually want when they ask "where do I rank for X".

Persisting Day-Over-Day Deltas

A rank tracker without history is just a daily snapshot. The persistence layer is one CSV per day plus a wide-format rankings.csv that joins yesterday to today:

def main():
    today = date.today().isoformat()
    yesterday = (date.today() - timedelta(days=1)).isoformat()

    # 1. Run today's scrape
    with open("keywords.csv") as f:
        rows = list(csv.DictReader(f))

    today_results = []
    for row in rows:
        try:
            res = track_one(row["keyword"], row["country"], row["language"], row["target_domain"])
            today_results.append({**row, "date": today, "position": res["position"]})
        except Exception as e:
            today_results.append({**row, "date": today, "position": None, "error": str(e)})

    # 2. Write today's snapshot
    today_path = f"history/{today}.csv"
    os.makedirs("history", exist_ok=True)
    with open(today_path, "w", newline="") as f:
        w = csv.DictWriter(f, fieldnames=["date", "keyword", "country", "target_domain", "position"])
        w.writeheader()
        for r in today_results:
            w.writerow({k: r.get(k) for k in w.fieldnames})

    # 3. Compute deltas against yesterday
    y_positions = {}
    y_path = f"history/{yesterday}.csv"
    if os.path.exists(y_path):
        with open(y_path) as f:
            for r in csv.DictReader(f):
                key = (r["keyword"], r["country"], r["target_domain"])
                y_positions[key] = int(r["position"]) if r["position"] else None

    with open("rankings.csv", "w", newline="") as f:
        w = csv.writer(f)
        w.writerow(["keyword", "country", "target", "today", "yesterday", "delta"])
        for r in today_results:
            key = (r["keyword"], r["country"], r["target_domain"])
            today_p = r["position"]
            y_p = y_positions.get(key)
            delta = (y_p - today_p) if (today_p and y_p) else None
            w.writerow([r["keyword"], r["country"], r["target_domain"], today_p, y_p, delta])


if __name__ == "__main__":
    main()

delta is positive when the page moved up (lower position number = better rank), negative when it moved down, and None for new entrants and exits. Sorting rankings.csv by delta ascending puts the biggest losers at the top, which is the order the SEO lead reviews on Monday morning.

The history/<date>.csv files build up a permanent record. After 30 days you have a month of position data per keyword per country that you can pivot into trend charts, compute volatility-per-keyword, or feed into a churn-risk model for client retention reporting.

Two extensions of this storage layout pay for themselves quickly. First, store the full top-20 organic block per keyword per day in a separate serps/<date>.jsonl file — not just the target's position. This costs a few MB per day for a 5,000-keyword set, and it is what makes the competitor-share-of-voice analysis possible later without re-scraping. The position-only CSV is the daily reporting artifact; the JSONL serp store is the analytical warehouse. Second, track a featured_snippet_owner column in the daily CSV. Featured snippets sit above position 1 and absorb 30–60% of organic clicks for the queries that have one. Losing a featured snippet drops a page from position 0 to position 1 — which the position tracker reports as "no change", but which represents a real CTR loss the client will feel in their analytics.

Scaling Beyond a Single Keyword List

Tracking 50 money keywords daily is one cron job. Tracking 5,000 keywords across 8 countries — which is what an agency portfolio actually looks like — needs two scaling adjustments.

Parallelize the submit step. The script above issues one job at a time. With 5,000 keywords that becomes the wall-clock bottleneck even though each individual job completes in 5–10 seconds. Submit jobs in parallel using concurrent.futures.ThreadPoolExecutor with 10–20 workers; the polling loop inside each submit_and_wait handles the rest. A 5,000-keyword run drops from 14 hours sequential to under 30 minutes parallel.

Tier the schedule. Run the money-keyword set every day at the same time. Run the tracking-set keywords once a week, partitioned by day-of-week so one-seventh runs each day — that smooths the request load and prevents the cron job from creating a daily spike. The discovery set runs monthly.

For very large agency portfolios, the bulk-submission pattern lets you hand the whole keyword list to the platform in one POST and let the scheduler distribute it across the proxy pool, instead of orchestrating it client-side. The full bulk pattern is covered in the web scraping API guide.

Scaling with LogPose

For agencies and in-house SEO teams running this at portfolio scale, the rate-limit and proxy-rotation problem is the wall most DIY rank trackers hit by week six. Google starts CAPTCHA-walling the source IP after a few hundred queries from the same address per day, and the manual proxy-rotation logic — pool maintenance, IP-warm-up, country-bound residential rotation — quietly becomes more code than the rank tracker itself.

The LogPose web scraping API handles the proxy rotation, country-bound IP selection, and CAPTCHA recovery transparently behind the /api/v1/search/google/search endpoint shown above. Set country=us and a US residential IP is selected automatically; set country=de and the next request comes from a German one. The rank-tracker pipeline stays at ~50 lines of Python because the infrastructure is on the API side. The same endpoint is what powers Google News and Google Shopping rank tracking — switch the path from /search/google/search to /search/google/news or /search/google/shopping to track positions in those verticals without changing the rest of the pipeline.

Common Mistakes

• Reporting position from a personalized SERP. If you scrape from your own logged-in IP without setting country explicitly, you are reporting your own browser's personalized rank — not the canonical rank a new user in that country sees. Always pass country and use a residential IP in that country.
• Counting sponsored slots in the position number. Ads above the organic block do not push your organic position from 1 to 4. Keep ads and organic strictly separate; report against organic only.
• Tracking too deep. Scraping pages=10 to get positions 1–100 costs 5× more than pages=2 and produces noisier data — position 80 vs 90 is statistical noise, not a ranking change. Cap at pages=2 for rank tracking; use deeper scrapes only when you specifically need long-tail discovery.
• Running the scrape at the same UTC time globally. Google's index refreshes are timezone-staggered. Schedule each country's scrape to run between 02:00 and 06:00 in that country's primary timezone for the most stable SERP snapshot.
• Ignoring the Cloudflare 100-second edge timeout. api.logposervices.com sits behind Cloudflare, so a job that takes 100+ seconds returns a 524 to your client even though the job continues server-side. Always poll for status; never expect a synchronous response on a big page count.

Legality and Ethics

Scraping the public Google SERP for rank tracking is what every commercial SEO platform has done for two decades. Google's Terms of Service forbid automated access to the underlying APIs without a key, but the rendered HTML of google.com/search?q=... is public, indexable by other search engines, and not behind authentication. US case law (hiQ Labs v. LinkedIn, 9th Cir. 2022) confirms that scraping public web data is not a CFAA violation. Rank tracking is reading what is already visible to any visitor; the data is never republished as a product, and no Google user account is involved. The risk profile is materially different from scraping authenticated services, and is the same risk profile every Ahrefs and SEMrush customer has been comfortable with since 2010.

Get Started

1. Sign up at logposervices.com and generate an API key under Tool → API Keys.
2. export LOGPOSE_API_KEY=lp_xxxxxxx
3. Create a keywords.csv with your tracked terms and run the pipeline above against it.
4. Schedule it via cron: 0 4 * * * /usr/bin/python3 /opt/rank-tracker/main.py >> /var/log/ranks.log 2>&1

Related reading: How to scrape Google Maps for local business leads for the local-pack equivalent of organic rank tracking, the web scraping API guide for the broader DIY-vs-managed comparison, and How to enrich business leads with emails, phones, and socials for the post-rank-tracking step of converting organic visibility into outbound pipeline.

External: Google Search Central, Search Console documentation, hiQ Labs v. LinkedIn.