ByteByteGo

I’ve been writing the system design newsletter for 12 months. Here are the 5 most popular ones: 👇 1. From 0 to Millions: A Guide to Scaling Your App 2. A Crash Course in Caching 3. API Architectural Styles 4. How does ChatGPT work? 5. 8 Data Structures That Power Your Databases Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/3FEGliw .

Do you think PassKey is safer than a password? . . Google announced PassKey support for both Android and Chrome recently. Passkeys, also backed by Apple and Microsoft, is claimed to be a significantly 𝐬𝐚𝐟𝐞𝐫 𝐫𝐞𝐩𝐥𝐚𝐜𝐞𝐦𝐞𝐧𝐭 for passwords. The diagram below shows how PassKeys work. 🔹Step 1 - create PassKeys The end-user needs to confirm the account information and present their credentials (face ID, touch ID, etc.). A private key is generated based on the public key provided by the website. The private key is stored on the device. 🔹Step 2 - sign in with PassKeys on devices When the user tries to sign in to a website, they use the generated private key. Just select the account information and present the credentials to unlock the private key. Consequently, there is no risk of password leakage since no passwords are stored in the websites' databases. Passkeys are built on 𝐢𝐧𝐝𝐮𝐬𝐭𝐫𝐲 𝐬𝐭𝐚𝐧𝐝𝐚𝐫𝐝𝐬, and it works across different platforms and browsers - including Windows, macOS and iOS, and ChromeOS, with a 𝐮𝐧𝐢𝐟𝐨𝐫𝐦 𝐮𝐬𝐞𝐫 𝐞𝐱𝐩𝐞𝐫𝐢𝐞𝐧𝐜𝐞. -- Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social   #systemdesign #coding #interviewtips  .

What are the 𝐝𝐢𝐟𝐟𝐞𝐫𝐞𝐧𝐜𝐞𝐬 between Redis and Memcached? The diagram below illustrates the key differences. The advantages on data structures make Redis a good choice for: 🔹 Recording the number of clicks and comments for each post (hash) 🔹 Sorting the commented user list and deduping the users (zset) 🔹 Caching user behavior history and filtering malicious behaviors (zset, hash) 🔹 Storing boolean information of extremely large data into small space. For example, login status, membership status. (bitmap) -- Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social   #systemdesign #coding #interviewtips  .

How can Cache Systems go wrong? The diagram below shows 4 typical cases where caches can go wrong and their solutions. 1. Thunder herd problem This happens when a large number of keys in the cache expire at the same time. Then the query requests directly hit the database, which overloads the database. There are two ways to mitigate this issue: one is to avoid setting the same expiry time for the keys, adding a random number in the configuration; the other is to allow only the core business data to hit the database and prevent non-core data to access the database until the cache is back up. 2. Cache penetration This happens when the key doesn’t exist in the cache or the database. The application cannot retrieve relevant data from the database to update the cache. This problem creates a lot of pressure on both the cache and the database. To solve this, there are two suggestions. One is to cache a null value for non-existent keys, avoiding hitting the database. The other is to use a bloom filter to check the key existence first, and if the key doesn’t exist, we can avoid hitting the database. 3. Cache breakdown This is similar to the thunder herd problem. It happens when a hot key expires. A large number of requests hit the database. Since the hot keys take up 80% of the queries, we do not set an expiration time for them. 4. Cache crash This happens when the cache is down and all the requests go to the database. There are two ways to solve this problem. One is to set up a circuit breaker, and when the cache is down, the application services cannot visit the cache or the database. The other is to set up a cluster for the cache to improve cache availability. Over to you: Have you met any of these issues in production?   –  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social   #systemdesign #coding #interviewtips  .

Kubernetes Periodic Table    A comprehensive visual guide that demystifies the key building blocks of this powerful container orchestration platform.    This Kubernetes Periodic Table sheds light on the 120 crucial components that make up the Kubernetes ecosystem.    Whether you're a developer, system administrator, or cloud enthusiast, this handy resource will help you navigate the complex Kubernetes landscape.    –  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social   #systemdesign #coding #interviewtips  .

IBM MQ -> RabbitMQ -> Kafka ->Pulsar, How do message queue architectures evolve?    🔹 IBM MQ  IBM MQ was launched in 1993. It was originally called MQSeries and was renamed WebSphere MQ in 2002. It was renamed to IBM MQ in 2014. IBM MQ is a very successful product widely used in the financial sector. Its revenue still reached 1 billion dollars in 2020.    🔹 RabbitMQ  RabbitMQ architecture differs from IBM MQ and is more similar to Kafka concepts. The producer publishes a message to an exchange with a specified exchange type. It can be direct, topic, or fanout. The exchange then routes the message into the queues based on different message attributes and the exchange type. The consumers pick up the message accordingly.    🔹 Kafka  In early 2011, LinkedIn open sourced Kafka, which is a distributed event streaming platform. It was named after Franz Kafka. As the name suggested, Kafka is optimized for writing. It offers a high-throughput, low-latency platform for handling real-time data feeds. It provides a unified event log to enable event streaming and is widely used in internet companies.    Kafka defines producer, broker, topic, partition, and consumer. Its simplicity and fault tolerance allow it to replace previous products like AMQP-based message queues.    🔹 Pulsar  Pulsar, developed originally by Yahoo, is an all-in-one messaging and streaming platform. Compared with Kafka, Pulsar incorporates many useful features from other products and supports a wide range of capabilities. Also, Pulsar architecture is more cloud-native, providing better support for cluster scaling and partition migration, etc.    There are two layers in Pulsar architecture: the serving layer and the persistent layer. Pulsar natively supports tiered storage, where we can leverage cheaper object storage like AWS S3 to persist messages for a longer term.    Over to you: which message queues have you used?    –  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social   #systemdesign #coding #interviewtips  .

Payment Systems Around The World Series (Part 1): Unified Payments Interface (UPI) in India. . . What’s UPI? UPI is an instant real-time payment system developed by the National Payments Corporation of India. It accounts for 60% of digital retail transactions in India today and still growing. UPI = payment markup language + standard for interoperable payments Let's take a look at how it works. 𝐑𝐞𝐠𝐢𝐬𝐭𝐫𝐚𝐭𝐢𝐨𝐧 1. Bob wants to open an account and provides his phone number +91 12345678 2. Bob performs OTP (One-Time Password) phone verification 3. Bob sets up VPA (Virtual Payment Address) bobaxis 4. Bob’s payment app creates VPA with the acquiring bank 5. The acquiring bank returns with VPA 6. The payment app returns VPA to Bob 𝐋𝐢𝐧𝐤 𝐭𝐨 𝐁𝐚𝐧𝐤 𝐀𝐜𝐜𝐨𝐮𝐧𝐭 7. Bob wants to link his SBI bank account with VPA bob at the axis. The request is forwarded to NPCI (National Payments Corporation of India). 8. NPCI acts as a switch between acquiring banks and issuing banks. It resolves the account detail from VPA with different issuing banks. 9. Bob authenticates with account details and sets the PIN, which is used for 2FA. This goes all the way to the issuing bank. Reference: “UPI 101” by Setu   –  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social   #systemdesign #coding #interviewtips  .

Foreign Exchange Explained. Suppose Bob (the buyer) needs to pay 100 USD to Alice (the seller), and Alice can only receive EUR. The diagram below illustrates the process. 1. Bob sends 100 USD via a third-party payment provider. In our example, it is Paypal. The money is transferred from Bob’s bank account (Bank B) to Paypal’s account in Bank P1. 2. Paypal needs to convert USD to EUR. It leverages the foreign exchange provider (Bank E). Paypal sends 100 USD to its USD account in Bank E. 3. 100 USD is sold to Bank E’s funding pool. 4. Bank E’s funding pool provides 88 EUR in exchange for 100 USD. The money is put into Paypal’s EUR account in Bank E. 5. Paypal’s EUR account in Bank P2 receives 88 EUR. 6. 88 EUR is paid to Alice’s EUR account in Bank A. Now let’s take a close look at the foreign exchange (forex) market. It has 3 layers: 🔹 Retail market. Funding pools are parts of the retail market. To improve efficiency, Paypal usually buys a certain amount of foreign currencies in advance. 🔹 Wholesale market. The wholesale business is composed of investment banks, commercial banks, and foreign exchange providers. It usually handles accumulated orders from the retail market. 🔹 Top-level participants. They are multinational commercial banks that hold lots of money from different countries. When Bank E’s funding pool needs more EUR, it goes upward to the wholesale market to sell USD and buy EUR. When the wholesale market accumulates enough orders, it goes upward to top-level participants. Steps 3.1-3.3 and 4.1-4.3 explain how it works. If you have any questions, please leave a comment. What foreign currency did you find difficult to exchange? And what company have you used for foreign currency exchange?   –  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social   #systemdesign #coding #interviewtips  .

Understanding Database Types To make the best decision for our projects, it is essential to understand the various types of databases available in the market. We need to consider key characteristics of different database types, including popular options for each, and compare their use cases.   –  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social   #systemdesign #coding #interviewtips  .

4 Key Ways for a web server to provide real-time update An HTTP server cannot automatically initiate a connection to a browser. As a result, the web browser is the initiator. What should we do next to get real-time updates from the HTTP server? Both the web browser and the HTTP server could be responsible for this task. 🔹Web browsers do the heavy lifting: short polling or long polling. With short polling, the browser will retry until it gets the latest data. With long polling, the HTTP server doesn’t return results until new data has arrived. 🔹HTTP server and web browser cooperate: WebSocket or SSE (server-sent event). In both cases, the HTTP server could directly send the latest data to the browser after the connection is established. The difference is that SSE is uni-directional, so the browser cannot send a new request to the server, while WebSocket is fully-duplex, so the browser can keep sending new requests. Over to you: of the four solutions (long polling, short polling, SSE, WebSocket), which ones are commonly used, and for what use cases? –  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/bbg-social   #systemdesign #coding #interviewtips  .