Labeled Datasets
Tech Link Quality Corpus — 215,061 verified web page records
215,061 quote-verified URL metadata records from arXiv, GitHub, Wikipedia and Stack Overflow — with AI quality verdicts, keywords and documented methodology. JSONL.
What you get: 1 file · 51.49 MB · ZIP
See purchase optionsProduct description
# Tech Link Quality Corpus — 215,061 verified web page records
Delivery: ZIP package with the JSONL data file (zstd-compressed), LICENSE, buyer guide (README) and the judge methodology (METHODOLOGY.md). Instant download.
## What you get
Metadata for 215,061 English-language technical web pages — the quote-verified layer of a 225,852-record corpus. Every record: URL, domain, source type, an LLM-judge verdict — "substantive page vs stub" — with a short written rationale, and 10–15 keywords.
Records also carry page_topic_hint — a rule-extracted page topic (arXiv categories like "arxiv: Computer Science > Machine Learning"). It is filled for ~22% of records; the rest carry a technical placeholder. For topical filtering use keywords, present on every record.
One JSONL record (trimmed to key fields; full records have 16 fields: pid, slug, theme, url, domain, tap, source, track, worthy, reason, keywords, phrases, entities, brands_products, quotes_verified, page_topic_hint):
```json
{
"pid": "kv_cache_llm_05509c8977",
"url": "https://arxiv.org/abs/2607.20495",
"domain": "arxiv.org",
"tap": "arxiv",
"worthy": true,
"reason": "A substantive arXiv paper page with a full abstract on workload-aware caching...",
"keywords": ["KV cache", "LLM inference", "eviction policy"],
"page_topic_hint": "arxiv: Computer Science > Machine Learning",
"quotes_verified": true
}
```
## What it is good for
- Training a page-quality classifier: 215k labeled examples (substantive vs stub/boilerplate/404/ad) with per-record rationale
- Seed list for RAG / AI-search crawling: live, substantive technical URLs, pre-filtered
- URL prospecting filters for SEO tooling
## Sources and composition
- 1,899 distinct domains in the dataset; top sources by records: github.com 28.9%, arxiv.org 28.5%, en.wikipedia.org 21.3%, stackoverflow.com 17.7%, datatracker.ietf.org 2.3%, others ~1.5% (EUR-Lex, ICLR, Ethresear.ch and misc tech blogs)
- 100% https, 99.98% unique URLs, snapshot: November 2026
- Quality checks: every record cross-verified against the page text quotes_verified: true in all 215,061); records with broken judge–page binding excluded; live spot-check of 25 random URLs: 21/25 alive, all substantive
## What is NOT included (by design)
- Page texts (metadata only)
- Verbatim quotes from pages (copyright hygiene)
- The original harvest-query field — it is a search seed, not a classification. For topical filtering use keywords (on every record); page_topic_hint adds arXiv categories where available (~22% of records)
## Honest limitations
- Verdicts are machine-generated (GLM-5.3-flash, temperature 0.2, documented prompt); the prompt targets a technical corpus, but the judge accepted some substantive non-technical pages (~15–20% of worthy=true) — filterable by reason text, and the methodology to re-classify is included
- 12,000-character input cap per page: annotations describe the head of each page
- page_topic_hint is filled for ~22% of records (mostly arXiv categories); the rest carry a technical placeholder
- Links age; snapshot dated November 2026
## Format
- clean-verified.jsonl.zst: 215,061 quote-verified records — the data file in this package (51.5 MB compressed); the package also includes LICENSE, buyer guide (README) and METHODOLOGY.md
- The full layer (225,852 records, each with the quotes_verified flag) can be provided on request — ask in the product chat
- JSONL, one record per line, zstd frame; opens with any language's zstd binding
- Free sample of 4,000 records on Hugging Face (link in the product card)
What you get
1 file · 51.49 MB
- getly-package.zipZIP · 51.49 MB
Labeled Datasets
Tech Link Quality Corpus — 215,061 verified web page records
215,061 quote-verified URL metadata records from arXiv, GitHub, Wikipedia and Stack Overflow — with AI quality verdicts, keywords and documented methodology. JSONL.
Free trial from huggingface.co
Tags
Frequently asked questions
Do I get access instantly?
Yes. This is a digital product - you can download it right after checkout, straight from your library.
Can I use it for commercial projects?
Personal use only. Use it for yourself - not in work you sell or hand to a client - and do not resell or redistribute the file.
What's your refund policy?
You can request a refund within 30 days. If something isn't right, request it from your order and the Getly team reviews it.
What file formats and sizes will I get?
You'll download 1 file - ZIP - 51.49 MB in total.
Do I get free updates?
Yes - when the seller ships a new version, you can re-download the latest files from your library at no extra cost. Check the changelog for recent updates.
Price
$39.00



