Step 1: Install Dependencies

npm install node-cron axios mongoose dotenv

• node-cron: Handles task scheduling using cron expressions.
• axios: For making HTTP requests to fetch data.
• mongoose: Provides a schema-based solution to work with MongoDB.
• dotenv: Allows secure handling of environment variables.

Step 2: Define the MongoDB Schema

// src/models/UrlConfig.js
const mongoose = require("mongoose");
const urlConfigSchema = new mongoose.Schema({
 name: { type: String, required: true }, // A friendly name for the job
 url: { type: String, required: true },  // The URL to fetch data from
 schedule: { type: String, required: true, default: "*/15 * * * *" }, // Cron schedule
 isActive: { type: Boolean, default: true }, // Whether the job is active
 lastRun: Date,    // Timestamp of the last run
 nextRun: Date,    // Calculated timestamp of the next run
 status: { 
   type: String, 
   enum: ["idle", "running", "completed", "failed"], 
   default: "idle" // Job state
 },
}, { timestamps: true }); // Automatically adds createdAt and updatedAt
module.exports = mongoose.model("UrlConfig", urlConfigSchema);

Explanation:
The schema defines job properties:

• name: A descriptive name for the task.
• url: The endpoint to be fetched.
• schedule: Cron expression dictating job timing.
• isActive: Determines if the job is currently running.
• status: Tracks job state (e.g., running or failed).

Step 3: Initialize the Cron Manager

const cron = require("node-cron");
const axios = require("axios");
const UrlConfig = require("../models/UrlConfig");
const jobs = new Map(); // Stores all active cron jobs
async function initializeCronManager() {
 try {
   await loadActiveConfigurations(); // Load configurations on startup
   cron.schedule("* * * * *", loadActiveConfigurations); // Reload configs every minute
   console.log("Cron Manager initialized");
 } catch (error) {
   console.error("Initialization failed:", error);
 }
}

Explanation:

• jobs: A Map to store active cron jobs, with job IDs as keys.
• initializeCronManager: Starts by loading jobs from the database and sets up a task to refresh configurations every minute.

Step 4: Load Active Configurations

async function loadActiveConfigurations() {
 const activeConfigs = await UrlConfig.find({ isActive: true }); // Fetch active jobs
 const activeIds = new Set(activeConfigs.map(config => config.id));
 for (const jobId of jobs.keys()) {
   if (!activeIds.has(jobId)) stopJob(jobId); // Stop jobs no longer active
 }
 for (const config of activeConfigs) {
   await startOrUpdateJob(config); // Start or update active jobs
 }
}

Explanation:

1. Fetch Active Jobs: Retrieves all jobs marked as isActive: true from the database.
2. Stop Inactive Jobs: Compares running jobs with the database and stops those no longer active.
3. Start/Update Jobs: Ensures active jobs are scheduled with the correct configuration.

Step 5: Start or Update Jobs

async function startOrUpdateJob(config) {
 const { id, schedule, name, url } = config;
 if (jobs.get(id)?.schedule === schedule) return; // Skip if schedule hasn't changed
 stopJob(id); // Stop existing job if it exists
 const job = cron.schedule(schedule, () => executeJob(config)); // Schedule new job
 jobs.set(id, { job, schedule }); // Add to active jobs
 console.log(`Started job: ${name}`);
}
 

Explanation:

• Checks if the job already exists with the same schedule. If unchanged, it skips scheduling.
• Stops existing jobs before starting a new schedule.
• Uses cron.schedule to create a job and adds it to the jobs map.

Step 6: Stop Jobs

function stopJob(jobId) {
 const jobInfo = jobs.get(jobId);
 if (jobInfo) {
   jobInfo.job.stop(); // Stops the job
   jobs.delete(jobId); // Removes it from the map
   console.log(`Stopped job: ${jobId}`);
 }
}

Explanation:

• Locates the job by its ID in the jobs map.
• Stops and removes the job from the list of active jobs.

Step 7: Execute Jobs

async function executeJob(config) {
 const { id, name, url, schedule } = config;
 try {
   await UrlConfig.findByIdAndUpdate(id, { status: "running", lastRun: new Date() }); // Update status
   const response = await axios.get(url); // Fetch URL
   console.log(`Response from ${name}:`, response.data);
   await UrlConfig.findByIdAndUpdate(id, { 
     status: "completed", 
     nextRun: calculateNextRun(schedule) // Update next run time
   });
 } catch (error) {
   console.error(`Execution failed for ${name}:`, error);
   await UrlConfig.findByIdAndUpdate(id, { status: "failed" });
 }
}

Explanation:

• Marks the job as running and logs the lastRun time.
• Uses axios to fetch data from the specified URL.
• Updates the job's status to completed or failed based on the outcome.

Step 8: Calculate Next Run Time

function calculateNextRun(schedule) {
 return cron.parseExpression(schedule).next().toDate(); // Determine next execution
}

Explanation:

• Uses cron.parseExpression to calculate the next execution time based on the schedule.

Step 9: Start the Application

require("dotenv").config();
const mongoose = require("mongoose");
const { initializeCronManager } = require("./src/services/cronManager");
async function startApplication() {
 try {
   await mongoose.connect(process.env.MONGODB_URI);
   console.log("Connected to MongoDB");
   await initializeCronManager(); // Start cron manager
   console.log("Application started");
 } catch (error) {
   console.error("Startup failed:", error);
   process.exit(1); // Exit on failure
 }
}
startApplication();

Explanation:

• Connects to MongoDB using credentials from .env.
• Initializes the cron manager to start scheduling jobs.

Key Features

1. Dynamic Scheduling: Jobs are managed in real-time via the database.
2. Error Handling: Logs failed jobs and their responses.
3. Scalable Design: Perfect for periodic tasks like API data fetching.
4. Decoupled Logic: Clear separation between configuration, job execution, and cron management.

With this system, you can dynamically control all your cron jobs by simply updating the MongoDB database. It’s ideal for automating tasks like background monitoring or periodic API calls.