Ultimate Browsing
FreeBypass web access restrictions for data extraction.
Free · Opens the source repo
What Ultimate Browsing does
Ultimate Browsing is a specialized skill designed to facilitate web access when conventional methods fail. It is particularly useful for developers and data engineers who encounter obstacles such as Web Application Firewalls (WAF), 403 errors, or sites that require JavaScript rendering and authentication. This skill is structured into three tiers, allowing users to escalate their approach based on the complexity of the web access issue at hand.
The first tier, known as 'insane-search', utilizes headless extraction techniques to bypass common barriers. It employs tools like curl_cffi for TLS impersonation, yt-dlp for media metadata extraction, and Jina Reader for accessing public APIs. This tier is optimized for speed and efficiency, making it approximately ten times faster than traditional browser methods. Users can quickly extract text or data from blocked URLs without needing a browser UI, which is ideal for tasks such as scraping or fetching metadata from platforms like YouTube and Reddit.
For scenarios involving Chinese social media platforms or other specialized content, the second tier, 'agent-reach', provides access to platform-native APIs that outperform generic fetching methods. This tier is particularly valuable for accessing content from platforms like Douyin, Weibo, and Xiaohongshu, which often require specific handling that standard tools cannot provide. The skill allows for seamless integration with these platforms, enhancing data extraction capabilities.
Lastly, the third tier, 'Chrome stealth', is employed when real interaction is necessary. This includes tasks that require clicking, form submissions, or media playback. Utilizing a stealth version of Chromium, this tier can effectively navigate advanced bot detection systems, ensuring that users can interact with web applications as needed. Overall, Ultimate Browsing is a powerful tool for anyone needing to access and extract data from challenging web environments.
When to use it
Use this skill when standard web requests are blocked or when you need to interact with complex web pages that require JavaScript or authentication.
When not to use it
This skill is not suitable for simple web searches or standard data fetching; use web-search or webfetch instead.
What you can build with it
Extracting Data from Blocked URLs
When a URL is blocked due to WAF or other restrictions, use Tier 1 to quickly extract the necessary data without a browser.
Accessing Chinese Social Media Content
For platforms like Weibo or Douyin, leverage Tier 1.5 to utilize platform-native APIs for efficient data retrieval.
Interacting with Complex Web Applications
When you need to fill forms or capture screenshots, escalate to Tier 2 for real-time interaction using a stealth browser.
How to install Ultimate Browsing
View source1. Install with the skills CLI
npx skills add code-yeongyu/oh-my-openagent/ultimate-browsing --agent claude-code2. Or install it manually
Download the skill folder and drop it into ~/.claude/skills/ for all projects, or .claude/skills/ to scope it to one repo. Restart Claude Code so it picks up the new skill.
Anthropic's agentic coding CLI, and the reference implementation of Agent Skills. Drop a skill folder into ~/.claude/skills and Claude Code loads it automatically whenever a task matches the skill's description. Claude Code docs
Inside SKILL.md
Written by code-yeongyuUltimate Browsing
Escalation web access for tasks a normal browse or fetch cannot complete. Reach for this skill the moment a page is blocked (WAF / 403 / Cloudflare), needs JS rendering, hides behind a login, or lives on a platform a generic fetcher cannot read. Escalate only when the cheaper tier cannot do the job:
Tier 1 — insane-search (headless extraction + WAF bypass) -> Tier 1.5 — agent-reach (platform-native APIs, esp. Chinese platforms) -> Tier 2 — Chrome stealth (real interaction via CloakBrowser + agent-browser).
PHASE 0 — ROUTE FIRST (MANDATORY)
User request
|
+- extract text/data from a URL --------------------- TIER 1 insane-search
+- URL blocked / 403 / Cloudflare / WAF ------------- TIER 1 insane-search
+- YouTube/Vimeo/TikTok subtitles or metadata ------- TIER 1 insane-search (yt-dlp)
+- read an article / blog / Reddit / HN / arXiv ----- TIER 1 insane-search
|
+- Chinese platform (xhs/douyin/weibo/bilibili/v2ex/wechat) TIER 1.5 agent-reach
+- podcast transcript / stock forum ----------------- TIER 1.5 agent-reach
+- Twitter feed / LinkedIn profile / GitHub via CLI - TIER 1.5 agent-reach
|
+- Tier 1/1.5 returned empty or partial ------------- TIER 2 Chrome stealth
+- click / fill form / scroll / interact ------------ TIER 2 Chrome stealth
+- screenshot / render / play video ----------------- TIER 2 Chrome stealth
+- login session across pages / inject cookies ------ TIER 2 Chrome stealth
+- test web app / QA / dogfood ---------------------- TIER 2 Chrome stealth
|
+- simple search query ------------------------------ NOT this skill (use web-search)
Read the matching reference before acting: references/insane-search/README.md, references/agent-reach/README.md, or references/chrome-stealth.md.
Tier 1 — insane-search (headless extraction)
When: content extraction, blocked-URL bypass, media metadata — no browser UI needed.
Why first: ~10x faster than a browser, no process spin-up; handles most "fetch this blocked page" requests via curl_cffi TLS impersonation, yt-dlp (1858 sites), Jina Reader, official public APIs, mobile URL transforms, and a Playwright real-Chrome fallback. The engine lives inside this skill at engine/ and is invoked as a module.
# Core command — auto-detects WAF, runs the full fetch grid (run from the skill dir):
python3 -m engine "https://example.com/blocked-page"
# add --selector "<CSS>" for positive-proof validation, --device auto|desktop|mobile,
# --trace to inspect every attempt, --json for machine-readable output.
# YouTube subtitles / metadata (no browser):
yt-dlp --write-sub --write-auto-sub --sub-lang "en,ko" --skip-download -o "/tmp/%(id)s" "<URL>"
# Reddit / HN / Bluesky / arXiv etc. use official public endpoints — see the Phase 0 index in
# references/insane-search/README.md (Twitter syndication, Reddit .json, HN Firebase, ...).
The full engine harness (rules R1-R7, the Phase 0 official-API index, the no-site-name rule, and the references/insane-search/*.md deep-dives for TLS, Playwright routing, Naver, media, etc.) is in references/insane-search/README.md. Read it before tuning the engine or adding a WAF profile.
Escalate to Tier 1.5 or Tier 2 when
- The target is a Chinese / social platform with a native reader -> Tier 1.5.
- insane-search returns empty/partial, or the page needs JS interaction, a screenshot, a persistent login, or media playback -> Tier 2.
Tier 1.5 — agent-reach (platform-native readers)
When: the target is a platform with a first-class API/CLI that beats generic fetching — especially Chinese platforms that stealth browsers still cannot reach cleanly. Several channels are zero-config (Douyin, Weibo via Jina, V2EX, Reddit, Jina Reader, RSS, YouTube); others need a one-time auth you supply via environment variables if you have access.
| Category | Platforms | Entry |
|---|---|---|
| social | xhs (Xiaohongshu), douyin, weibo, bilibili, V2EX, Reddit, Twitter/X | references/agent-reach/social.md |
| web | Jina Reader, WeChat articles, RSS | references/agent-reach/web.md |
| video | YouTube, Bilibili, podcast transcripts, Douyin video | references/agent-reach/video.md |
| career | references/agent-reach/career.md | |
| dev | GitHub (gh CLI) | references/agent-reach/dev.md |
| search | Exa AI | references/agent-reach/search.md |
mcporter call 'douyin.parse_douyin_video_info(url: "<URL>")' # douyin, zero-config
curl -s "https://r.jina.ai/https://weibo.com/<uid>/<pid>" # weibo via Jina
yt-dlp --dump-json "<bilibili-url>" # Bilibili (overseas: add --cookies-from-browser)
curl -s "https://www.v2ex.com/api/topics/hot.json" # V2EX public API
Routing table, per-platform auth (set TWITTER_* env vars, gh auth login, a transcription key — only if you have access), rate-limit notes, and known version quirks are in references/agent-reach/README.md.
Tier 2 — Chrome stealth (real interaction)
When: real interaction is needed (clicks, forms, screenshots, video, persistent login), or Tier 1/1.5 failed.
CloakBrowser is a stealth Chromium with source-level fingerprint patches that passes Cloudflare Turnstile, FingerprintJS, BrowserScan, and 30+ detectors; agent-browser is the CDP automation CLI that drives it. Both are runtime-installed tools (not vendored here). Full setup, version pins, launch flow, cookie login, and cross-platform notes are in references/chrome-stealth.md.
# 1. Launch CloakBrowser with CDP on :9242 (see chrome-stealth.md for install + venv).
# 2. CloakBrowser launches tabless — open the first tab via CDP before any agent-browser command:
curl -s -X PUT "http://127.0.0.1:9242/json/new?https://example.com"
# 3. Drive it with agent-browser over CDP:
agent-browser --cdp 9242 snapshot -i # interactive elements (@eN refs)
agent-browser --cdp 9242 click @e3
agent-browser --cdp 9242 screenshot out.png
agent-browser --cdp 9242 close
Cookie login (cross-platform)
scripts/extract_cookies.py reads cookies from a local Chromium-family or Firefox-family browser and optionally injects them into the running CDP session. It resolves browser profile paths and decrypts cookie values per-OS (macOS Keychain, Linux libsecret, Windows DPAPI):
# Extract cookies to a file:
mkdir -p ~/.local/state/omo-cookies
python3 scripts/extract_cookies.py --browser chrome --domain youtube.com --output ~/.local/state/omo-cookies/youtube.cookies.json
# Extract and inject into the running CDP session:
python3 scripts/extract_cookies.py --browser chrome --domain youtube.com --inject --cdp 9242
Cookie export files are written with owner-only 0600 permissions. Do not place live auth cookies in shared temp directories or commit them to a repo. Cookie injection sends values to CDP over stdin rather than argv. Cookies apply on next navigation — reload after injecting. Google services use fingerprint-bound tokens that may not transfer across browser profiles. Full detail in references/chrome-stealth.md.
Reference docs
| File | When to read |
|---|---|
| references/insane-search/README.md | Tier-1 engine harness (R1-R7, Phase 0 API index, no-site-name rule) + its *.md deep-dives |
| references/agent-reach/README.md | Tier-1.5 routing table, platform auth, per-category *.md |
| references/chrome-stealth.md | Tier-2 CloakBrowser + agent-browser install, CDP flow, version pins, cookie login |
Environment variables
CLOAK_CDP_PORT=9242 # CloakBrowser CDP port (default 9242)
AGENT_BROWSER_USER_AGENT="..." # override UA to hide HeadlessChrome
AGENT_BROWSER_HEADED=1 # show the browser window
# agent-reach auth: set the channel-specific env vars from each tool's docs only if you have access
# insane-search needs no env vars — it auto-installs deps on first run
Anti-patterns
- Do NOT launch Chrome stealth for plain text extraction — use Tier 1.
- Do NOT pass an
--init-scriptfor the webdriver flag — CloakBrowser already patches it at source; the only required override is--user-agent. - Do NOT run agent-browser before creating the first tab via
curl -X PUT .../json/new— CloakBrowser launches tabless. - Do NOT use vanilla Chrome when stealth is needed — always CloakBrowser.
- Do NOT forget to
closethe session when done. - Do NOT inject cookies without reloading the page.
- Do NOT hardcode site domains/selectors into
engine/**orwaf_profiles.yaml— runtime hints only (see the no-site-name rule in the insane-search reference).
Frequently asked questions about Ultimate Browsing
Similar skills
WinMD API Search
Easily find and explore Windows desktop APIs.
WebMCPify
Transform any web app into an agent-ready platform.
Phoenix Tracing
Instrument LLM applications with OpenInference tracing.
Foundry Hosted Agent CopilotKit
Guidance for developing agentic web apps on Azure.
Power Automate Foundation
Connect AI agents to Power Automate seamlessly.
Power Automate Flow Builder
Efficiently build and deploy Power Automate flows programmatically.