Skip to main content

Testing & Deployment

Learning Objectives​

In this lab, you'll:

  • Deploy your complete Intelligence Collector to production
  • Perform comprehensive testing of all features
  • Validate data quality
  • Set up monitoring and alerting for production operations

Time Estimate​

10 minutes

Deployment Checklist​

Pre-Deployment Validation​

Before deploying to production, ensure you have:

  • ✅ KV Namespace Configured - THREAT_INTEL namespace created and bound
  • ✅ Cron Schedule Active - 15-minute collection schedule configured
  • ✅ Error Handling Tested - All failure scenarios handled gracefully
  • ✅ API Endpoints Working - All REST endpoints responding correctly
  • ✅ Status Page Functional - Real-time monitoring dashboard operational
  • ✅ Whitelist Protection Active - Cloudflare IPs automatically protected

Step 1: Final Deployment​

Deploy your complete Intelligence Collector:

# Ensure you're in the intelligence-collector directory
cd intelligence-collector

# Run final TypeScript check
npm run type-check

# Deploy to production
npx wrangler deploy

# Verify deployment
npx wrangler status

Expected Output:

✨ Successfully deployed
🌍 Available at: https://intelligence-collector.YOUR-SUBDOMAIN.workers.dev

Step 2: Comprehensive API Testing​

Test all API endpoints systematically:

2.1 Trigger Initial Data Collection (only works locally)​

# Run locally
npm run dev

# Trigger first collection manually
curl -s "http://localhost:8787/cgi/handler/scheduled?cron=*+*+*+*+*"

# Wait for collection to complete (watch logs)
# Should see: "Enhanced collection complete: X active threats..."

2.2 Test Threat Intelligence Endpoints​

# Test paginated threat listing
curl -s "https://intelligence-collector.YOUR-SUBDOMAIN.workers.dev/ips?limit=5" | jq '.pagination'

# Test filtering by confidence level
curl -s "https://intelligence-collector.YOUR-SUBDOMAIN.workers.dev/ips?confidence=high&limit=3" | jq '.data[].confidence_level'

2.3 Test System Statistics​

# Get comprehensive system statistics
curl -s "https://intelligence-collector.YOUR-SUBDOMAIN.workers.dev/stats" | jq '.data.collection'

# Check data quality metrics
curl -s "https://intelligence-collector.YOUR-SUBDOMAIN.workers.dev/stats" | jq '.data.data_quality'

Step 3: Production Monitoring Setup​

3.1 Set Up Log Monitoring​

# Monitor production logs in real-time
npx wrangler tail --format=pretty

# Look for these key log patterns:
# ✅ "Enhanced collection complete: N active threats"
# ✅ "Whitelist filtering complete: {...}"
# ❌ "Collection failed:" (should not appear)
# ❌ "ERROR:" (investigate immediately)

Validation Checklist​

After completing all tests, verify:

  • ✅ All API endpoints respond correctly
  • ✅ Status page shows operational status
  • ✅ Threat intelligence collection working
  • ✅ Whitelist protection active
  • ✅ Data quality metrics > 80%
  • ✅ Performance metrics acceptable
  • ✅ Error handling graceful
  • ✅ Monitoring setup complete

Troubleshooting Common Issues​

Issue: No data collection​

# Check cron trigger configuration
npx wrangler publish --dry-run

# Check logs for errors
npx wrangler tail

Issue: API returning errors​

# Check KV namespace binding
npx wrangler kv namespace list

# Verify environment configuration
cat wrangler.jsonc

Issue: Poor performance​

# Check collection duration
curl -s "https://intelligence-collector.YOUR-SUBDOMAIN.workers.dev/stats" | jq '.data.collection.duration_ms'

# If > 10000ms, review source timeout settings

Production Deployment Complete! 🎉​

What You've Accomplished​

You've successfully built and deployed a production-ready Intelligence Collector with:

  • Multi-source threat intelligence aggregation
  • Intelligent scoring and confidence levels
  • Whitelist-first security architecture
  • Real-time monitoring and status page
  • Comprehensive REST API
  • Advanced data processing and validation
  • Performance monitoring and optimization

Key Production Features​

  1. Scalability - Runs on Cloudflare's global edge network
  2. Reliability - Graceful failure handling and automatic recovery
  3. Performance - Sub-second API responses with intelligent caching
  4. Security - Whitelist protection and input validation
  5. Observability - Comprehensive monitoring and alerting
  6. Maintainability - Self-documenting APIs and status pages

Next Phase​

Your Intelligence Collector is now ready to integrate with the Enforcement Engine in Phase 2! The clean API design and standardized data formats will make integration seamless.

Ready for Phase 2? Head to the Enforcement Engine Implementation to build the automated policy enforcement system that will consume your threat intelligence.

Key Takeaways​

  1. Testing is Critical - Comprehensive testing prevents production issues
  2. Monitoring from Day One - Build observability into your system architecture
  3. Documentation Matters - Good documentation enables team collaboration
  4. Performance by Design - Measure and optimize from the beginning
  5. Security First - Validate inputs and protect against edge cases