[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"work:queueing-system:en":3},[4,42,45,48],{"type":5,"items":6},"toc",[7,11,14,17,20,23,26,30,33,36,39],{"id":8,"level":9,"title":10},"502-bad-gateway",2,"502 Bad Gateway",{"id":12,"level":9,"title":13},"then-i-ll-build-one","Then I'll Build One",{"id":15,"level":9,"title":16},"choosing-the-api-stack","Choosing the API Stack",{"id":18,"level":9,"title":19},"storing-the-data","Storing the Data",{"id":21,"level":9,"title":22},"architecture","Architecture",{"id":24,"level":9,"title":25},"waiting-flow","Waiting Flow",{"id":27,"level":28,"title":29},"1-where-should-the-queue-ticket-be-checked",3,"1. Where should the queue ticket be checked?",{"id":31,"level":28,"title":32},"2-where-should-the-queue-ticket-be-stored","2. Where should the queue ticket be stored?",{"id":34,"level":28,"title":35},"3-how-should-traffic-be-controlled","3. How should traffic be controlled?",{"id":37,"level":9,"title":38},"after-launch","After Launch",{"id":40,"level":9,"title":41},"closing-thoughts","Closing Thoughts",{"type":43,"html":44},"html","\u003Cp>\u003Cimg src=\"__WORK_ASSET__502-bad-gateway.webp\" alt=\"\" loading=\"lazy\">\u003C\u002Fp>\n\u003Ch2 id=\"502-bad-gateway\">502 Bad Gateway\u003C\u002Fh2>\n\u003Cp>The site went down during an event.\u003C\u002Fp>\n\u003Cp>There was nothing I could do but watch the 502 responses for 30 minutes until the traffic subsided. I felt helpless.\u003C\u002Fp>\n\u003Cp>There were constraints in the environment at the time, so I could neither scale the site’s servers nor introduce a commercial solution. I was left thinking that the site might not have gone down if we had had a queue.\u003C\u002Fp>\n\u003Ch2 id=\"then-i-ll-build-one\">Then I’ll Build One\u003C\u002Fh2>\n\u003Cp>So I decided to build one.\u003C\u002Fp>\n\u003Cp>I set two main goals.\u003C\u002Fp>\n\u003Cul>\n\u003Cli>Protect the servers from traffic that arrives all at once\u003C\u002Fli>\n\u003Cli>Cost less than a commercial solution\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Cp>Once I had the goals, the possible approach began to narrow.\u003C\u002Fp>\n\u003Cul>\n\u003Cli>\n\u003Cp>Protect the servers from traffic that arrives all at once\u003Cbr>\n→ \u003Cstrong>Build a high-performance queueing system on AWS!\u003C\u002Fstrong>\u003C\u002Fp>\n\u003C\u002Fli>\n\u003Cli>\n\u003Cp>Cost less than a commercial solution\u003Cbr>\n→ \u003Cstrong>Make it possible to turn the servers on and off!\u003C\u002Fstrong>\u003C\u002Fp>\n\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Cp>Next came the design. I did not know how to build a queueing service, so I studied one reference at a time, including materials from \u003Ca href=\"https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=MTSn93rNPPE\" target=\"_blank\" rel=\"noreferrer\">Woowahan Tech\u003C\u002Fa>, \u003Ca href=\"https:\u002F\u002Fdev.gmarket.com\u002F46\" target=\"_blank\" rel=\"noreferrer\">Gmarket\u003C\u002Fa>, and the \u003Ca href=\"https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=3pO9GJ4zndE\" target=\"_blank\" rel=\"noreferrer\">Nexon Developers Conference\u003C\u002Fa>. As expected, the internet had plenty of useful material shared by developers who had faced the same problem.\u003C\u002Fp>\n\u003Cp>I learned from their experience and narrowed down the technology and implementation approach.\u003C\u002Fp>\n\u003Ch2 id=\"choosing-the-api-stack\">Choosing the API Stack\u003C\u002Fh2>\n\u003Cp>A large-scale sequential processing system does not have CPU-intensive computation, but it performs a great deal of short I\u002FO while communicating with Redis. When user requests arrive in a short burst, the cost of thread occupancy and context switching increases.\u003C\u002Fp>\n\u003Cp>Java 21 virtual threads could also have been a good option. However, for a structure where requests rise sharply and Redis I\u002FO is heavy, I decided that Spring WebFlux was more suitable because it could be designed around non-blocking I\u002FO from the start.\u003C\u002Fp>\n\u003Cp>Another option was Node.js with Express, but I chose Java because I was more familiar with it and could develop faster.\u003C\u002Fp>\n\u003Cp>Most events start at a scheduled time. I therefore planned to use AWS Auto Scaling so that the system would scale only when needed and reduce costs. Unexpected situations could still occur, so I built a deployment pipeline that created a golden image with every deployment.\u003C\u002Fp>\n\u003Ch2 id=\"storing-the-data\">Storing the Data\u003C\u002Fh2>\n\u003Cp>I chose Redis without much hesitation.\u003C\u002Fp>\n\u003Cp>Because Redis is an in-memory database, it is well suited to high concurrency and frequent changes in ranking. Its Sorted Set also provides score-based ordering and rank lookup, which makes the sequential processing problem easier to solve.\u003C\u002Fp>\n\u003Cp>It can find how many people are ahead of me and my position in \u003Ccode>O(log N)\u003C\u002Fcode>, so the lookup is very fast.\u003C\u002Fp>\n\u003Cp>The system could also have been built with a traditional DBMS or Kafka. But it is difficult for a DBMS to provide the same throughput and low latency as Redis, while Kafka alone makes it difficult to query a user’s current position in real time.\u003C\u002Fp>\n\u003Cp>To secure enough performance, I made the queueing API depend only on Redis. I also disabled both RDB and AOF to minimize latency. Queue data is temporary, so losing it was acceptable. Instead, I placed master and replica instances across three VMs for availability.\u003C\u002Fp>\n\u003Cp>\u003Cimg src=\"__WORK_ASSET__redis-cluster.webp\" alt=\"\" loading=\"lazy\">\u003C\u002Fp>\n\u003Cp>If one server fails, one master and one replica fail with it. But because they are arranged across servers, the setup can withstand a single failure. Deploying it on AWS with each node in a different Availability Zone would provide some fault tolerance. Given the traffic level and cost at the time, I considered this an acceptable trade-off.\u003C\u002Fp>\n\u003Cp>(With a larger budget, I would like to try a six-VM cluster, or simply use AWS ElastiCache for peace of mind.)\u003C\u002Fp>\n\u003Ch2 id=\"architecture\">Architecture\u003C\u002Fh2>\n\u003Cp>I also set up the following services for the queueing system. This was the first completed architecture.\u003C\u002Fp>\n\u003Cul>\n\u003Cli>An admin system for managing queues\u003C\u002Fli>\n\u003Cli>A database for storing queue information\u003C\u002Fli>\n\u003Cli>CI\u002FCD (Jenkins)\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Cp>\u003Cimg src=\"__WORK_ASSET__arch-first.webp\" alt=\"\" loading=\"lazy\">\u003C\u002Fp>\n\u003Ch2 id=\"waiting-flow\">Waiting Flow\u003C\u002Fh2>\n\u003Cp>I kept the queueing flow simple.\u003C\u002Fp>\n\u003Cul>\n\u003Cli>Issue a queue ticket.\u003C\u002Fli>\n\u003Cli>If the user can enter immediately, send them to the requested page. If they need to wait, send them to the waiting page.\u003C\u002Fli>\n\u003Cli>Periodically check their position and whether they can enter.\u003C\u002Fli>\n\u003Cli>When the wait ends, send them to the page they originally tried to access.\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Cp>While users wait, the admission scheduler checks the Redis queue at regular intervals and changes only as many users as the server can handle to the \u003Ccode>ready\u003C\u002Fcode> state. Users who finish waiting then move to the page they originally tried to access.\u003C\u002Fp>\n\u003Cp>The implementation itself was not difficult… but I had three questions.\u003C\u002Fp>\n\u003Cp> \u003Cstrong>1. Where should the queue ticket be checked?\u003C\u002Fstrong>\u003Cbr>\n \u003Cstrong>2. Where should the queue ticket be stored?\u003C\u002Fstrong>\u003Cbr>\n \u003Cstrong>3. How should traffic be controlled?\u003C\u002Fstrong>\u003C\u002Fp>\n\u003Ch3 id=\"1-where-should-the-queue-ticket-be-checked\">1. Where should the queue ticket be checked?\u003C\u002Fh3>\n\u003Cp>Admission through the queue has to be fair.\u003C\u002Fp>\n\u003Cp>Someone who has waited should be admitted, while someone who has not should be sent to the back of the queue.\u003C\u002Fp>\n\u003Cp>In Next.js, I split entry into two paths and implemented queue validation for each.\u003C\u002Fp>\n\u003Cp> a. Initial entry (refresh, new tab, and so on) → Check in Next.js middleware\u003Cbr>\n b. Internal navigation through the router → Check in the browser\u003C\u002Fp>\n\u003Ch4 id=\"a-initial-entry\">a. Initial entry\u003C\u002Fh4>\n\u003Cp>I checked initial entry in middleware because of the page structure at the time.\u003C\u002Fp>\n\u003Cp>If the queue were checked in \u003Ccode>useEffect\u003C\u002Fcode> after the page mounted, another \u003Ccode>useEffect\u003C\u002Fcode> could run first because of the page structure. If that effect sent an API request, it would add load to the server before the queue check had finished. The queue would fail at its job of protecting the server during a traffic spike.\u003C\u002Fp>\n\u003Cpre>\u003Ccode>Initial entry\n  → Check the queue in middleware\n    → Entry allowed: Send a completed ticket and enter immediately\n    → Waiting required: Redirect to the waiting page\n\u003C\u002Fcode>\u003C\u002Fpre>\u003Cp>If no wait is required, the middleware puts a completed ticket that allows immediate entry in the response header. The client detects it during initialization and allows entry. If a wait is required, the middleware redirects the user to the waiting page.\u003C\u002Fp>\n\u003Ch4 id=\"b-internal-navigation-through-the-router\">b. Internal navigation through the router\u003C\u002Fh4>\n\u003Cp>After the initial entry, the router checks the queue immediately before navigating to another page.\u003C\u002Fp>\n\u003Cp>If immediate entry is available, it takes the user to the requested page. Otherwise, the user waits. Once the wait ends, a callback runs and takes the user to the page they originally requested.\u003C\u002Fp>\n\u003Cpre>\u003Ccode>Attempt internal navigation\n→ Check the queue in the browser\n  → Entry allowed: Move to the original page\n  → Waiting required: Show the waiting page\n\u003C\u002Fcode>\u003C\u002Fpre>\u003Ch4 id=\"wouldn-t-it-be-easier-to-check-everything-in-middleware\">Wouldn’t it be easier to check everything in middleware?\u003C\u002Fh4>\n\u003Cp>I could have checked every navigation in middleware. But with that implementation, every queue validation request would pass through the Next.js middleware and then be forwarded to the queueing API. When event traffic spiked, this would add unnecessary load to Node. The queueing API would handle the heavy logic, but the Node server would still have to relay each request.\u003C\u002Fp>\n\u003Ch4 id=\"what-if-someone-tries-queue-tickets-at-random\">What if someone tries queue tickets at random?\u003C\u002Fh4>\n\u003Cp>The queue ticket number was exposed in the browser API response and followed a pattern to some degree, so a brute-force attack could predict another user’s ticket number. I therefore combined the ticket number with a secret to create a hash and validated it on the server. This prevented an attacker from trying other users’ queue tickets at random and misusing them.\u003C\u002Fp>\n\u003Ch3 id=\"2-where-should-the-queue-ticket-be-stored\">2. Where should the queue ticket be stored?\u003C\u002Fh3>\n\u003Cp>The next decision was where to store and read the ticket.\u003Cbr>\nThere were four candidates. In the end, I stored it in React state.\u003C\u002Fp>\n\u003Cul>\n\u003Cli>Cookie\u003C\u002Fli>\n\u003Cli>localStorage\u003C\u002Fli>\n\u003Cli>sessionStorage\u003C\u002Fli>\n\u003Cli>React state\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Cp>Cookies and localStorage are shared between tabs on the same origin.\u003Cbr>\nIf a user opened several tabs and entered the queue, an existing queue ticket could conflict with the ticket in another tab. I therefore ruled them out.\u003C\u002Fp>\n\u003Cp>sessionStorage was another option, but the browser’s session restoration feature could leave an old ticket in place after the browser was closed and reopened.\u003C\u002Fp>\n\u003Cp>A queue ticket must be used once per page and then discarded, so I decided to manage it in React state.\u003Cbr>\nIt is not shared between tabs, and reopening a tab reloads the page, which best matched the requirements at the time.\u003C\u002Fp>\n\u003Ch3 id=\"3-how-should-traffic-be-controlled\">3. How should traffic be controlled?\u003C\u002Fh3>\n\u003Cp>The next question was how many people to admit at once.\u003C\u002Fp>\n\u003Cp>At first, I used a fixed arrival rate based on the idea of backpressure, admitting a configured number of users every second.\u003C\u002Fp>\n\u003Cpre>\u003Ccode>Admit N users per second\n  → Limit the sudden influx at the start of an event\n\u003C\u002Fcode>\u003C\u002Fpre>\u003Cp>The limitation was that this method controlled only the number of incoming users. It did not account for the load created by users already on the page. If users left quickly, admitting a fixed number every second was enough. But if they stayed for a long time, the number of concurrent users on the page kept growing even when the admission rate remained constant.\u003C\u002Fp>\n\u003Cp>Although the queue controlled arrivals, it did not know how many users remained on the page. When users accumulated on a particular page, the load would rise again.\u003C\u002Fp>\n\u003Cp>So I changed the control criterion from “how many users should enter each second” to “how many users are currently on the page.”\u003C\u002Fp>\n",{"type":46,"name":47},"directive","traffic-capacity-animation",{"type":43,"html":49},"\u003Cp>I treated each page as a room and set a maximum capacity for each room.\u003C\u002Fp>\n\u003Cp>When the current occupancy reaches the maximum, the system blocks new entries. When users leave the page and space becomes available, it admits the same number of waiting users from the queue.\u003C\u002Fp>\n\u003Cp>This made it possible to see how many users were in each section and control traffic based on the actual load on a particular page.\u003C\u002Fp>\n\u003Cp>\u003Cimg src=\"__WORK_ASSET__admin.webp\" alt=\"\" loading=\"lazy\">\u003C\u002Fp>\n\u003Ch2 id=\"after-launch\">After Launch\u003C\u002Fh2>\n\u003Cp>After introducing the queue, we ran several events. It handled large events without difficulty, including one where 100,000 people entered the queue within three minutes. As an engineer, watching a system I built work as intended may be one of the most tense and exhilarating experiences I have had.\u003C\u002Fp>\n\u003Cp>\u003Cimg src=\"__WORK_ASSET__waiting-page.webp\" alt=\"Waiting page during an event\" loading=\"lazy\">\u003C\u002Fp>\n\u003Cp>(This image is a reconstruction based on the actual production screen.)\u003C\u002Fp>\n\u003Ch2 id=\"closing-thoughts\">Closing Thoughts\u003C\u002Fh2>\n\u003Cp>Taking on something I had never done before was not easy. We worked in an environment where systems were usually operated by purchasing and introducing commercial solutions, so developing one in-house was itself a considerable burden. There was a lot of trial and error, but I achieved the goal of keeping the system up during events. I then learned that building the feature was not the end of the work.\u003C\u002Fp>\n\u003Cp>Once the system was in operation, recording business metrics and making logs visible also proved important for providing a stable service. We had no data showing the maximum number of people who had waited, and no logs for finding the cause when an error occurred. There was also a time when we noticed a Redis Cluster failure far too late.\u003C\u002Fp>\n\u003Cp>I began filling those gaps one at a time: recording business metrics in InfluxDB, viewing server and infrastructure metrics through Prometheus and Grafana, and using Alertmanager to detect failures quickly.\u003C\u002Fp>\n\u003Cp>This is the current architecture after those changes.\u003C\u002Fp>\n\u003Cp>\u003Cimg src=\"__WORK_ASSET__arch-final.webp\" alt=\"\" loading=\"lazy\">\u003C\u002Fp>\n\u003Cp>There is still a lot to do. The CI\u002FCD setup needs improvement, and security weaknesses need to be addressed. Solving one problem brought another to the surface. Implementing features and building a sustainable system seem to be different kinds of work. Since I built this system, it is work I will have to continue taking responsibility for.\u003C\u002Fp>\n\u003Cp>Thank you for reading this long article.\u003C\u002Fp>\n"]