Performing OSINT with SpiderFoot

Overview

SpiderFoot is an open-source OSINT automation tool with 200+ modules that integrates with data sources for threat intelligence and attack surface mapping. This skill uses the SpiderFoot REST API and CLI (sf.py/spiderfoot-cli) to create and manage scans, select modules by use case (footprint, investigate, passive), parse structured results for domains, IPs, email addresses, leaked credentials, and DNS records, and generate target intelligence profiles.

Prerequisites

• SpiderFoot 4.0+ installed or SpiderFoot HX cloud account
• Python 3.8+ with requests library
• SpiderFoot server running on default port 5001
• Optional: API keys for VirusTotal, Shodan, HaveIBeenPwned modules

Steps

1. Connect to SpiderFoot REST API or use CLI interface
2. Create a new scan with target specification (domain, IP, email, name)
3. Select scan modules by use case (all, footprint, investigate, passive)
4. Monitor scan progress via API polling
5. Retrieve and parse scan results by data element type
6. Extract key findings: subdomains, IPs, emails, leaked credentials
7. Generate structured OSINT intelligence report

Expected Output

JSON report containing OSINT findings organized by data type (domains, IPs, emails, credentials, DNS records), module source attribution, and target profile summary with risk indicators.

Verification Criteria

Confirm successful execution by validating:

• [ ] All prerequisite tools and access requirements are satisfied
• [ ] Each workflow step completed without errors
• [ ] Output matches expected format and contains expected data
• [ ] No security warnings or misconfigurations detected
• [ ] Results are documented and evidence is preserved for audit

Compliance Framework Mapping

This skill supports compliance evidence collection across multiple frameworks:

• SOC 2: CC7.1 (Monitoring), CC7.2 (Anomaly Detection)
• ISO 27001: A.6.1 (Threat Intelligence), A.16.1 (Security Incident Management)
• NIST 800-53: PM-16 (Threat Awareness), RA-3 (Risk Assessment), SI-5 (Security Alerts)
• NIST CSF: ID.RA (Risk Assessment), DE.AE (Anomalies & Events)

Claw GRC Tip: When this skill is executed by a registered agent, compliance evidence is automatically captured and mapped to the relevant controls in your active frameworks.

Deploying This Skill with Claw GRC

Agent Execution

Register this skill with your Claw GRC agent for automated execution:

# Install via CLI
npx claw-grc skills add performing-osint-with-spiderfoot

# Or load dynamically via MCP
grc.load_skill("performing-osint-with-spiderfoot")

Audit Trail Integration

When executed through Claw GRC, every step of this skill generates tamper-evident audit records:

• SHA-256 chain hashing ensures no step can be modified after execution
• Evidence artifacts (configs, scan results, logs) are automatically attached to relevant controls
• Trust score impact — successful execution increases your agent's trust score

Continuous Compliance

Schedule this skill for recurring execution to maintain continuous compliance posture. Claw GRC monitors for drift and alerts when re-execution is needed.