O*NET Occupation Data Scraper: 3 Practical Use Cases

Direct answer

Scrape O*NET OnLine (US Dept. of Labor) occupation data: SOC lookup, keyword search, browse by Job Zone or Bright Outlook. Tasks, skills, wages, growth outlook, related occupations. The most practical way to use this scraper is to map standard occupational classification records directly into software workflows: enriching ATS job posts with standardized tasks, powering career portal exploration paths, or feeding skill taxonomies into resume matching systems. The step-by-step guides below show how to configure each scenario, run focused extractions, and format the results into clear operational deliverables.

Practical use cases

These use cases come from ONET Occupation Data Scraper's published documentation. Each is expanded into an operating pattern so the ONET Occupation Data Scraper output has a purpose beyond collection.

Use case 1: HR tech & ATS platforms

Outcome: enrich job requisitions with standardized SOC task/skill data.

Question to answer: Once hard constraints are applied, which opportunities remain genuinely worth a closer look?

Configure: Start with jobZone (Preparation/education level to browse. Job Zone 1 has no standalone browse bucket on ONET OnLine - it is combined with Job Zone 2.), jobFamily (ONET Job Family (SOC major occupational group) to browse.), socCodes (ONET-SOC occupation codes to look up, format ##-####.## (e.g. 15-1252.00 for Software Developers). Find codes via mode=search first.). Use the narrowest ONET Occupation Data Scraper values that represent the real task, keep the first result set small, and record why each selected input matters to the ONET Occupation Data Scraper outcome.

Working method: Filter out hard mismatches using only documented fields before reading any description closely, then rank the survivors by preference fit and flag any with missing required information.

Deliverable: Create a shortlist grouped into strong fit, possible fit, and missing-information, each entry tied back to the field that decided its group. Include the ONET Occupation Data Scraper source identifier and the collected fields behind every ONET Occupation Data Scraper decision.

Stop condition: Pause when a listing's current status cannot be confirmed from a documented field, or the constraint check relies on the title text. Fix the ONET Occupation Data Scraper question, comparison rule, or configuration before expanding the ONET Occupation Data Scraper run.

Use case 2: Workforce development

Outcome: identify Bright Outlook occupations for training program design.

Question to answer: Applying the documented rule as written, which records clearly pass, which clearly fail, and which need a human call?

Configure: Start with brightOutlookCategory (Bright Outlook occupations are projected to grow rapidly, have many job openings, or are new/emerging.), brightOutlookOnly (Only keep occupations tagged Bright Outlook. Applies to search and all browse modes.), socCodes (ONET-SOC occupation codes to look up, format ##-####.## (e.g. 15-1252.00 for Software Developers). Find codes via mode=search first.). Use the narrowest ONET Occupation Data Scraper values that represent the real task, keep the first result set small, and record why each selected input matters to the ONET Occupation Data Scraper outcome.

Working method: Fix the accepted, uncertain, and excluded definitions before the first record is reviewed, then apply the rule consistently and log the field behind every classification so a second pass can check it.

Deliverable: Create a reviewed queue where every record keeps its raw form alongside the rule that accepted, excluded, or flagged it as uncertain. Include the ONET Occupation Data Scraper source identifier and the collected fields behind every ONET Occupation Data Scraper decision.

Stop condition: Pause when the same record is classified differently on a repeat pass, or a required field is missing often enough to change outcomes. Fix the ONET Occupation Data Scraper question, comparison rule, or configuration before expanding the ONET Occupation Data Scraper run.

Use case 3: Compensation benchmarking

Outcome: pull national and state wage percentiles per occupation.

Question to answer: After grouping fairly, which gaps are large enough to matter, and which are within normal variation?

Configure: Start with wageState (Optional. Also include state-level wage percentiles (10th/25th/median/75th/90th, hourly + annual) alongside the national figures. Leave blank for national-only.), socCodes (ONET-SOC occupation codes to look up, format ##-####.## (e.g. 15-1252.00 for Software Developers). Find codes via mode=search first.), searchQuery (Free-text keyword to search occupation titles, e.g. electrician, nurse, software developer.). Use the narrowest ONET Occupation Data Scraper values that represent the real task, keep the first result set small, and record why each selected input matters to the ONET Occupation Data Scraper outcome.

Working method: Lock the comparison rule before looking at any results, sort records into groups against that fixed rule, and keep a running note of anything that does not cleanly belong to a group.

Deliverable: Create a side-by-side comparison table with cohort labels, normalized fields, and a short note on what the gap implies. Include the ONET Occupation Data Scraper source identifier and the collected fields behind every ONET Occupation Data Scraper decision.

Stop condition: Pause when the comparison rule shifted mid-analysis, or a single record is skewing an entire cohort's average. Fix the ONET Occupation Data Scraper question, comparison rule, or configuration before expanding the ONET Occupation Data Scraper run.

Step-by-step extraction workflow

Follow these sequential steps to extract and operationalize O*NET data:

1. Identify the entry taxonomy mode: Determine whether your project begins with explicit SOC codes (bySocCode), keyword exploration (search), or structural taxonomy groups (browseByJobZone, browseByJobFamily, browseByCareerCluster, or browseByHotTechnology).
2. Run a minimal test batch: Launch a test query using maxItems: 5 to inspect the response structure and confirm that the returned fields align with downstream schema requirements.
3. Extract and map target properties: For full profiles, parse specific objects such as tasks[], softwareSkills[], wagePercentiles, and jobZone. For discovery modes, capture the socCode and onetUrl references.
4. Handle missing or pop-up attributes: Check optional arrays such as apprenticeshipTitles[] or partial softwareSkills[] listings, applying null values or follow-up queries where needed.
5. Load into target database: Ingest the validated, deduplicated records into your ATS database, search index, or candidate matching service.

Technical parameter configuration

The Actor supports several operational modes and filtering options:

• mode: Selects the retrieval method, including bySocCode, search, browseByJobZone, browseByBrightOutlook, browseByJobFamily, browseByIndustry, browseByCareerCluster, browseBySTEM, and browseByHotTechnology.
• socCodes: Array of target O*NET-SOC codes formatted as ##-####.## (for example, 15-1252.00).
• wageState: Two-letter US state code (such as CA or TX) to retrieve state-specific wage distributions alongside national percentiles.
• searchQuery: Free-text search term for locating occupations by title keywords (such as electrician or software developer).
• jobZone: Browse selector for preparation levels 1 through 5 (note that Zone 1 and Zone 2 are combined on O*NET OnLine).
• brightOutlookCategory: Filters for high-growth categories including rapid growth, numerous openings, or new and emerging fields.
• hotTechnology: Targets roles reporting active employer demand for specific software tools.
• titleContains: Case-insensitive title filter applied across search and browse modes.
• brightOutlookOnly: Boolean flag restricting results exclusively to Bright Outlook occupations.
• maxItems: Hard numeric ceiling on the total records returned.
• proxyConfiguration: Optional fallback settings, though public onetonline.org endpoints require no proxy or login.

Handling data anomalies and limitations

When designing production workflows around O*NET OnLine data, take these structural behaviors into account:

• Omitted empty fields: The Actor omits keys entirely when an occupation lacks specific attributes (such as apprenticeship titles or state wage percentiles). Upstream parsers should safely handle absent keys rather than expecting static default strings.
• Combined Job Zone buckets: Job Zone 1 does not exist as an independent browse category on the live website; browse queries combine Zones 1 and 2 under a single bucket.
• Software skill pagination: On the website, skill categories with extensive software examples hide additional items behind pop-up modals. The scraper extracts the visible software items from the initial page payload.
• Unauthenticated access: Because the scraper extracts structured data directly from the public onetonline.org site rather than the key-gated Web Services API, no account registration or proxy rotation is necessary.

Frequently asked questions

Does this scraper use the official O*NET Web Services API?

No. The scraper gathers data directly from the public onetonline.org website, eliminating the need for registered API keys or rate-limited credentials.

How are state-level wages retrieved?

When running mode: "bySocCode", pass a two-letter state abbreviation into the wageState parameter. The response will include 10th, 25th, 50th, 75th, and 90th percentile hourly and annual wages for that specific state alongside national metrics.

Can I browse occupations associated with specific software?

Yes. Set mode: "browseByHotTechnology" and define hotTechnology with a tool name such as Python, AWS, or Salesforce to extract all occupations reporting active employer demand for that skill.

Why are some optional fields missing from the output JSON?

O*NET OnLine omits data categories that do not apply to a given occupation. The scraper preserves this structure by omitting empty fields rather than populating dummy values.

Resources

• O*NET Occupation Data Scraper on Apify