How to use django to implement feature like Github Contribution Heatmap?

My ULibrary project want to have a feature like github's contribution heatmap.Green wall you know.

I alread have done a version using Nest.js in custom guard. It will figure out which role the request user belong then call loginHeatService to create today's record.

The process looks like:

flowchart
A[receive httpRequest] --> B{Authorization}
B -- YES --> C[check JWT Token]
C -- valid Token  --> D1[set request user]
D1 --> D[update user's Login Heatmap]
D --> G[service]
C -- invalid Token --> F[return 401 Unauthorized]
F --> E[Response]
B -- NO --> G[service]
G --> E[Response]

I know django will automatically set request user for us. Maybe we need django middleware.

So let's write a very simple middleware and test it:

MIDDLEWARE = [
    # CorsMiddleware should be placed as high as possible, especially before any middleware that can generate
    # responses such as Django’s CommonMiddleware or Whitenoise’s WhiteNoiseMiddleware. If it is not before,
    # it will not be able to add the CORS headers to these responses.
    "corsheaders.middleware.CorsMiddleware",
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
+    'ulibrary.middlewares.login_heatmap.login_heatmap_middleware' # Our middleware
]

It also have a class-based version:

class LoginHeatmapMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response
        # One-time configuration and initialization.

    def __call__(self, request):
        # Code to be executed for each request before
        # the view (and later middleware) are called.

        response = self.get_response(request)

        # Code to be executed for each request/response after
        # the view is called.

        return response

Here we cannot get the viewset's info so we don't know whether this request should be record in user' login heatmap.

Continue read document until we meet process_view:

Let add this process_view:

Good. Now we get view_func. So we can get view class.

Sad. When process_view called the request.user is not set(still AnonymousUser).Here request is WSGIRequest instance. Only drf's request instance will set user. And drf request instance will be created when:

So maybe we can easify it by using process_response. If we found response (here is drf response) have user we do our job.

New version of django doesn't have process_response function. We can use __call__() method and it will:

1. Calls self.process_request(request) (if defined).
2. Calls self.get_response(request) to get the response from later middleware and the view.
3. Calls self.process_response(request, response) (if defined).
4. Returns the response.

So our logic is:

temp_login_heatmap = dict()


class LoginHeatmapMiddleware(MiddlewareMixin):

    def __call__(self, request):
        # Code to be executed for each request before
        # the view (and later middleware) are called.

        response = self.get_response(request)
        # Code to be executed for each request/response after
        # the view is called.
        if request.user.is_authenticated:
            # check does user have today login heatmap
            logined = temp_login_heatmap.get(request.user.id) or False
            if logined:
                return response
            else:
                # create a login heatmap record for current user
                temp_login_heatmap[request.user.id] = True
        return response

After that we can change logic with databse operation:

from datetime import datetime
from django.utils.deprecation import MiddlewareMixin
from django.utils.timezone import make_aware

from ulibrary.models import LoginHeatmap
from ulibrary.serializers import LoginHeatmapSerializer


class LoginHeatmapMiddleware(MiddlewareMixin):

    def __call__(self, request):
        # Code to be executed for each request before
        # the view (and later middleware) are called.

        response = self.get_response(request)
        # Code to be executed for each request/response after
        # the view is called.
        if request.user.is_authenticated:
            # check does user have today login heatmap
            today = datetime.now().replace(hour=0, minute=0, second=0, microsecond=0)
            today_end = datetime.now().replace(hour=23, minute=59, second=59, microsecond=999)
            try:
                LoginHeatmap.objects.get(user=request.user.id,
                                         created__range=(make_aware(today), make_aware(today_end)))
                return response
            except LoginHeatmap.DoesNotExist:
                # create a login heatmap record for current user
                serializer = LoginHeatmapSerializer(data={"user": request.user.id})
                if serializer.is_valid():
                    serializer.save()
                    return response
        return response

Test Case

def test_query_or_create_user_login_heatmap(self):
    token, created = Token.objects.get_or_create(user=self.defaultUser)
    client = APIClient()
    client.credentials(HTTP_AUTHORIZATION='Token ' + token.key)

    # At first, user should not have login heatmap record
    today = datetime.now().replace(hour=0, minute=0, second=0, microsecond=0)
    today_end = datetime.now().replace(hour=23, minute=59, second=59, microsecond=999)
    count = LoginHeatmap.objects.filter(user=self.defaultUser.id,
                                        created__range=(make_aware(today), make_aware(today_end))).count()

    self.assertEqual(count, 0)

    response = client.get('/reading_histories/')
    self.assertEqual(response.status_code, status.HTTP_200_OK)

    # After access need authorization endpoint, login heatmap record should exist.
    count = LoginHeatmap.objects.filter(user=self.defaultUser.id,
                                        created__range=(make_aware(today), make_aware(today_end))).count()
    self.assertEqual(count, 1)

    # everyday will only exist one record
    # TODO how to makesure time will not past two day?
    response = client.get('/reading_histories/')

    self.assertEqual(response.status_code, status.HTTP_200_OK)
    count = LoginHeatmap.objects.filter(user=self.defaultUser.id,
                                        created__range=(make_aware(today), make_aware(today_end))).count()
    self.assertEqual(count, 1)

A query endpoint for frontend display

It works. But I think a better way is to only override get_queryset method:

We can also add some other feature like return recent one month or return recent one year data we don't need to talk about it in here.Just add some other condition in filter.

TODO Optimizations

This logic will trigger very frequency. How to reduce it?

---

Thanks for reading.