1. Web Ingestion Legality (ECTA Section 86 & Cybercrimes Act 19 of 2020)
Dubstrata's automated web crawlers (worker.py, regulatory_crawler.py) operate in strict compliance with Section 86 of the Electronic Communications and Transactions Act (ECTA) and the Cybercrimes Act (Act 19 of 2020):
- Public Web Endpoints Only: Crawlers scrape only un-authenticated, publicly accessible web documents and open sovereign registries.
- SSRF Protection: Internal network requests enforce strict Server-Side Request Forgery filtering (
is_ssrf_safe_url) to prevent querying private, loopback, or cloud metadata IP addresses. - Ethical Crawling & Robots.txt: Ingestion respects
robots.txtdirectives, implements HTTP Keep-Alive connection pooling, and adheres to ethical rate limits.
2. Prohibited Ingestion Targets
Subscribers may not direct Dubstrata's extraction tools toward:
- Authenticated financial banking portals or non-public clearinghouse interfaces.
- Healthcare portals containing private medical records (PHI).
- Endpoints protected by explicit digital rights management (DRM) or active law enforcement blockades.
- Any URI explicitly hosting malware, phishing kits, or illicit software.
3. Agent Swarm Rate Limits & Anti-DDoS Rules
While Dubstrata's distributed queuing engine scales horizontally, subscribers are prohibited from launching artificial concurrent sweeps (e.g. swarms issuing 10,000+ parallel requests against a single host):
Aggressive single-target flooding constitutes a Denial of Service attack. Abuse telemetry automatically triggers cryptographic wallet blacklisting and immediate API key revocation.
4. Reporting Security Vulnerabilities & Abuse
To report crawler rate anomalies or submit vulnerability disclosures, contact the Dubstrata Security Operations Center: