How to poll the backend at regular intervals

Intro#

Polling is a pull-based mechanism for fetching data from the server at regular intervals. It is useful when you want to refresh the UI on the frontend, based on real-time data from the backend.

This in contrast to a push-based mechanism, in which a long-lived SSE connection is kept open between the client and the server, and the server pushes updates to the client whenever necessary. Push-based mechanisms are more efficient than polling, and can be achieved using Datastar, but may be less desirable for some backends.

In PHP, for example, keeping long-lived SSE connections is fine for a dashboard in which users are authenticated, as the number of connections are limited. For a public-facing website, however, it is not recommended to open many long-lived connections, due to the architecture of most PHP servers.

Goal#

Our goal is to poll the backend at regular intervals (starting at 5 second intervals) and update the UI accordingly. The backend will determine changes to the DOM and be able to control the rate at which the frontend polls based on some criteria. For this example, we will simply output the server time, increasing the polling frequency to 1 second during the last 10 seconds of every minute. The criteria could of course be anything such as the number of times previously polled, the user’s role, load on the server, etc.

Demo#

Steps#

The data-on-interval attribute allows us to execute an expression at a regular interval. We’ll use it to send a GET request to the backend, and use the __duration modifier to set the interval duration.

<div id="time"
     data-on-interval__duration.5s="@get('/endpoint')"
></div>

In addition to the interval, we could also execute the expression immediately by adding .leading to the modifier.

<div id="time"
     data-on-interval__duration.5s.leading="@get('/endpoint')"
></div>

Most of the time, however, we’d just render the current time on page load using a backend templating language.

<div id="time"
     data-on-interval__duration.5s="@get('/endpoint')"
>
     {{ now }}
</div>

Now our backend can respond to each request with a datastar-merge-fragments event with an updated version of the element.

event: datastar-merge-fragments
data: fragments <div id="time" data-on-interval__duration.5s="@get('/endpoint')">
data: fragments     {{ now }}
data: fragments </div>

Be careful not to add .leading to the modifier in the response, as it will cause the frontend to immediately send another request.

Here’s how it might look using the SDKs.

(require
  '[starfederation.datastar.clojure.api :as d*]
  '[starfederation.datastar.clojure.adapter.http-kit :refer [->sse-response]])
  '[some.hiccup.library :refer [html]])

(import
  'java.time.format.DateTimeFormatter
  'java.time.LocalDateTime)

(def formatter (DateTimeFormatter/ofPattern "YYYY-MM-DD HH:mm:ss"))

(defn handle [ring-request]
   (->sse-response ring-request
     {:on-open
      (fn [sse]
        (d*/merge-fragment! sse
          (html [:div#time {:data-on-interval__duration.5s (d*/sse-get "/endpoint")}
                  (LocalDateTime/.format (LocalDateTime/now) formatter)])))}))

        (d*/close-sse! sse))}))

using StarFederation.Datastar.DependencyInjection;

app.MapGet("/endpoint", async (IDatastarServerSentEventService sse) =>
{
    var currentTime = DateTime.Now.ToString("yyyy-MM-dd hh:mm:ss");
    var fragment = $"""
                    <div id="time" data-on-interval__duration.5s="@get('/endpoint')">
                        {currentTime}
                    </div>
                    """;
    await sse.MergeFragmentsAsync(fragment);
});

import (
    "time"
    datastar "github.com/starfederation/datastar/sdk/go"
)

currentTime := time.Now().Format("2006-01-02 15:04:05")

sse := datastar.NewSSE(w, r)
sse.MergeFragments(fmt.Sprintf(`
    <div id="time" data-on-interval__duration.5s="@get('/endpoint')">
        %s
    </div>
`, currentTime))

import ServerSentEventGenerator
import ServerSentEventGenerator.Server.Snap -- or whatever is appropriate
import Data.Time ( getCurrentTime )
import Data.Text ( pack )

now <- getCurrentTime
let
  txt = mconcat [
    "<div id=\"time\" data-on-interval__duration.5s=\"@get('/endpoint')\">"
    , (pack . show) now
    , "</div>" ]
send $ mergeFragments txt def def def def

use starfederation\datastar\ServerSentEventGenerator;

$currentTime = date('Y-m-d H:i:s');

$sse = new ServerSentEventGenerator();
$sse->mergeFragments(`
    <div id="time"
         data-on-interval__duration.5s="@get('/endpoint')"
    >
        $currentTime
    </div>
`);

datastar = Datastar.new(request:, response:)

current_time = Time.now.strftime('%Y-%m-%d %H:%M:%S')

datastar.merge_fragments <<~FRAGMENT
    <div id="time"
         data-on-interval__duration.5s="@get('/endpoint')"
    >
        #{current_time}
    </div>
FRAGMENT

use datastar::prelude::*;
use chrono::Local;
use async_stream::stream;

let current_time = Local::now().format("%Y-%m-%d %H:%M:%S").to_string();

Sse(stream! {
    yield MergeFragments::new(
        format!(
            "<div id='time' data-on-interval__duration.5s='@get(\"/endpoint\")'>{}</div>",
            current_time
        )
    ).into();
})

import { createServer } from "node:http";
import { ServerSentEventGenerator } from "../npm/esm/node/serverSentEventGenerator.js";

const server = createServer(async (req, res) => {

  const currentTime = new Date().toISOString();
  ServerSentEventGenerator.stream(req, res, (sse) => {
    sse.mergeFragments(`
       <div id="time"
          data-on-interval__duration.5s="@get('/endpoint')"
       >
         ${currentTime}
       </div>
    `);
  });
});

const datastar = @import("datastar").httpz;
const zdt = @import("zdt");
const std = @import("std");

var tz_chicago = try zdt.Timezone.fromTzdata("America/Chicago", res.arena);
const datetime = try zdt.Datetime.fromISO8601("2006-01-02 15:04:05");
const current_time = try a_datetime.tzLocalize(.{ .tz = &tz_chicago });

var sse = try datastar.ServerSentEventGenerator.init(res);

sse.mergeFragments(
    std.fmt.allocPrint(
        res.arena,
        "<div id='time' data-on-interval__duration.5s='@get(\"/endpoint\")'>{s}</div>",
        .{current_time},
    ),
    .{},
);

Our second requirement was that the polling frequency should increase to 1 second during the last 10 seconds of every minute. To make this possible, we’ll calculate and output the interval duration based on the current seconds of the minute.

(require
  '[starfederation.datastar.clojure.api :as d*]
  '[starfederation.datastar.clojure.adapter.http-kit :refer [->sse-response]])
  '[some.hiccup.library :refer [html]])

(import
  'java.time.format.DateTimeFormatter
  'java.time.LocalDateTime)

(def date-time-formatter (DateTimeFormatter/ofPattern "YYYY-MM-DD HH:mm:ss"))
(def seconds-formatter (DateTimeFormatter/ofPattern "ss"))

(defn handle [ring-request]
  (->sse-response ring-request
    {:on-open
      (fn [sse]
        (let [now (LocalDateTime/now)
              current-time (LocalDateTime/.format now date-time-formatter)
              seconds (LocalDateTime/.format now seconds-formatter)
              duration (if (neg? (compare seconds "50"))
                         "5"
                         "1")]
          (d*/merge-fragment! sse
            (html [:div#time {(str "data-on-interval__duration." duration "s")
                              (d*/sse-get "/endpoint")}
                    current-time]))))}))

          (d*/close-sse! sse))}))

using StarFederation.Datastar.DependencyInjection;

app.MapGet("/endpoint", async (IDatastarServerSentEventService sse) =>
{
    var currentTime = DateTime.Now.ToString("yyyy-MM-dd hh:mm:ss");
    var currentSeconds = DateTime.Now.Second;
    var duration = currentSeconds < 50 ? 5 : 1;
    var fragment = $"""
                    <div id="time" data-on-interval__duration.{duration}s="@get('/endpoint')">
                        {currentTime}
                    </div>
                    """;
    await sse.MergeFragmentsAsync(fragment);
});

import (
    "time"
    datastar "github.com/starfederation/datastar/sdk/go"
)

currentTime := time.Now().Format("2006-01-02 15:04:05")
currentSeconds := time.Now().Format("05")
duration := 1
if currentSeconds < "50" {
    duration = 5
}

sse := datastar.NewSSE(w, r)
sse.MergeFragments(fmt.Sprintf(`
    <div id="time" data-on-interval__duration.%ds="@get('/endpoint')">
        %s
    </div>
`, duration, currentTime))

{-# LANGUAGE QuasiQuotes #-}
import ServerSentEventGenerator
import ServerSentEventGenerator.Server.Snap -- or whatever is appropriate
import Data.Time ( getCurrentTime )
import Data.Time.Format
import Data.Text ( pack, unpack )
import NeatInterpolation

now <- getCurrentTime
let
  formatted = pack $ formatTime defaultTimeLocale "%Y-%m-%d %H:%M:%S" now
  seconds   = formatTime defaultTimeLocale "%S" now
  duration  = pack $ if seconds < "50" then "5" else "1"
  message x y   =
    [trimming|
      <div id="time" data-on-interval__duration.${x}s="@get('/endpoint')">
        ${y}
       </div>
    |]
send $ mergeFragments (message duration formatted) def def def def

use starfederation\datastar\ServerSentEventGenerator;

$currentTime = date('Y-m-d H:i:s');
$currentSeconds = date('s');
$duration = $currentSeconds < 50 ? 5 : 1;

$sse = new ServerSentEventGenerator();
$sse->mergeFragments(`
    <div id="time"
         data-on-interval__duration.${duration}s="@get('/endpoint')"
    >
        $currentTime
    </div>
`);

datastar = Datastar.new(request:, response:)

now = Time.now
current_time = now.strftime('%Y-%m-%d %H:%M:%S')
current_seconds = now.strftime('%S').to_i
duration = current_seconds < 50 ? 5 : 1

datastar.merge_fragments <<~FRAGMENT
    <div id="time"
         data-on-interval__duration.#{duration}s="@get('/endpoint')"
    >
        #{current_time}
    </div>
FRAGMENT

use datastar::prelude::*;
use chrono::Local;
use async_stream::stream;

let current_time = Local::now().format("%Y-%m-%d %H:%M:%S").to_string();
let current_seconds = Local::now().second();
let duration = if current_seconds < 50 {
    5
} else {
    1
};

Sse(stream! {
    yield MergeFragments::new(
        format!(
            "<div id='time' data-on-interval__duration.{}s='@get(\"/endpoint\")'>{}</div>",
            duration,
            current_time,
        )
    ).into();
})

import { createServer } from "node:http";
import { ServerSentEventGenerator } from "../npm/esm/node/serverSentEventGenerator.js";

const server = createServer(async (req, res) => {
  const currentTime = new Date();
  const duration = currentTime.getSeconds > 50 ? 5 : 1;

  ServerSentEventGenerator.stream(req, res, (sse) => {
    sse.mergeFragments(`
       <div id="time"
          data-on-interval__duration.${duration}s="@get('/endpoint')"
       >
         ${currentTime.toISOString()}
       </div>
    `);
  });
});

const datastar = @import("datastar").httpz;
const zdt = @import("zdt");
const std = @import("std");

var tz_chicago = try zdt.Timezone.fromTzdata("America/Chicago", res.arena);
const datetime = try zdt.Datetime.fromISO8601("2006-01-02 15:04:05");
const current_time = try a_datetime.tzLocalize(.{ .tz = &tz_chicago });
const current_seconds = std.time.timestamp() % 60;
const duration = if (current_seconds < 50) 5 else 1;

var sse = try datastar.ServerSentEventGenerator.init(res);

sse.mergeFragments(
    std.fmt.allocPrint(
        res.arena,
        "<div id='time' data-on-interval__duration.{d}s='@get(\"/endpoint\")'>{s}</div>",
        .{ duration, current_time },
    ),
    .{},
);

Conclusion#

Using this approach, we not only end up with a way to poll the backend at regular intervals, but we can also control the rate at which the frontend polls based on whatever criteria our backend requires.