Frequently asked questions

We focus on publicly accessible commercial sources that can be collected reliably and responsibly. JavaScript-rendered pages, pagination, and complex public search flows are routine. We review access conditions, the requested fields, intended use, and source stability before confirming feasibility; we do not circumvent authentication or access controls.

Not at all. Our clients range from data engineers to business analysts to product managers. We handle all the technical implementation. You only need to know what data you want, from which websites, and how often. Everything else (extraction, normalization, QA, delivery, and maintenance) is handled by our team.

For most projects, we deliver a sample dataset within 3–7 business days of the discovery call. Production pipelines typically go live within 2–3 weeks depending on the number of sources and the technical complexity of the target sites. Rush timelines can sometimes be accommodated. Let us know your deadline on the discovery call.

Yes. A sample dataset is always part of the engagement process and is provided free of charge, with no obligation. The sample typically contains 500–2,000 records from your target sources and is used to validate data quality and schema before we agree on a production scope.

We support CSV, Excel (.xlsx), JSON, XML, Parquet, and custom schemas. Data can be delivered via SFTP, Amazon S3, Google Cloud Storage, Azure Blob, REST API endpoint, direct database insert, or Google Sheets. Most format and delivery combinations are available at no additional cost.

Delivery frequency ranges from real-time streaming to hourly, daily, weekly, or custom schedules. We'll recommend the appropriate frequency based on how often the source sites update and your operational requirements. More frequent delivery is always technically possible, the key question is whether the source data actually changes that often.

Yes. We support direct insert to PostgreSQL, MySQL, BigQuery, Snowflake, Redshift, and other SQL/cloud warehouse systems. Schema mapping, type coercion, and incremental upserts are all handled on our side. There is typically a one-time setup cost for database integration depending on complexity.

In many cases, yes. Historical data availability depends on the target sources. Where it exists (either on the live site via date filters or via public archives) we can extract it as a one-time backfill. Historical extraction is scoped and priced separately from ongoing pipeline pricing.

Yes. Cross-source record matching (also called entity resolution or SKU matching) is one of our core capabilities. We use GTIN, MPN, EAN, and fuzzy title/attribute matching to link product records across sources. Match confidence scores are included in the output so your team can set acceptance thresholds.

Our standard collection service is for publicly accessible data and does not use another person's account or defeat authentication. A source's official API, licensed feed, or customer-owned integration can be assessed separately when access is documented and authorized in writing.

We test public accessibility, page behavior, update cadence, rate limits, and representative volume before production. Collection uses responsible request pacing, retries, caching, and standard browser automation where appropriate. If a source cannot be collected reliably without crossing an access boundary, we exclude it from the scope and explain why.

Our monitoring systems detect structural and output changes and alert the engineering team automatically. The response target, communication path, and any backfill requirement are agreed during scoping, so you know what happens when a source changes.

Yes. We use headless browser automation (Chromium-based) for sites that require JavaScript rendering, dynamic content loading, or client-side routing. This applies to React, Angular, Vue, and other SPA frameworks. Performance is slightly lower than for static-rendered pages, so this is factored into our capacity planning and pricing.

The guarantee is tied to the written acceptance rules agreed for your project: schema, required fields, completeness thresholds, freshness, duplicate limits, and delivery format. A batch that misses those checks is held and corrected rather than knowingly shipped. If a covered issue is found after delivery, we investigate it and provide corrected output as part of the managed service. The guarantee covers the agreed dataset and checks; it is not a claim that third-party source websites will never change.

Accuracy is enforced at three levels: field-level validation during extraction (type checking, regex patterns, expected value ranges), cross-record consistency checks (deduplication, outlier detection, cross-source comparisons), and a final human review by a QA analyst on a random sample before each delivery. Every delivery is accompanied by a quality report showing pass rates per check.

Our standard validation suite runs 48 checks: null/missing field rates, data type conformance, URL validity, numeric range validation, date format consistency, duplicate detection, cross-source reconciliation where applicable, and category/taxonomy mapping accuracy. Clients can also define custom validation rules specific to their schema.

Our delivery reports include field completeness rates for every key field in the schema. If completeness falls below agreed thresholds, we flag it and investigate before delivery rather than shipping degraded data silently. For critical fields, we'll hold a delivery and communicate rather than let incomplete data flow into your systems.

Pricing is based on the number of approved sources, total data volume, collection frequency, source complexity and rate limits, number of fields, delivery format, QA requirements, and whether historical data is needed. We provide a written fixed-price proposal after discovery and feasibility review, so you know the scope and cost before committing.

Yes. Change requests are handled as part of the ongoing engagement. Adding fields, new sources, or changing delivery frequency is scoped and implemented, typically within a few days for minor changes. Your PM will advise on any impact to timeline or pricing before changes are made.

The legal position depends on the source, requested fields, intended use, contractual terms, and applicable jurisdiction. Pyronets focuses on publicly accessible data, reviews project boundaries during scoping, and declines work that requires circumventing access controls or collecting personal data without a lawful basis. Clients remain responsible for their own use of the delivered data and should obtain legal advice where appropriate.

We adapt to your preferred tools. Most clients use a combination of email and Slack for day-to-day communication, Basecamp or Notion for project documentation, and a weekly or bi-weekly check-in call for larger ongoing projects. Your PM will set up the right channels during onboarding.