1. High-Concurrency Architecture

In the previous article, we mentioned that there are many factors affecting planning in a high-concurrency system. This article will not discuss the writing style of the code, the language used, syntax, and other intricate factors, but will purely analyze from an infrastructure perspective.

Three-Tier Architecture (3-tier application)

If you have been involved in system architecture development and design, you must have heard of the classic three-tier architecture. In 1996, SUN Microsystems introduced Java, which was widely acclaimed and had a profound impact. One of the important reasons was that Java provided a multi-tier distributed module, no longer deploying the system on each terminal computer, which had a positive impact and application demand for data uniformity and data security of enterprises at that time.

The three-tier architecture divides the entire application into the presentation layer (UI), the business logic layer (BLL), and the data access layer (DAL). Users communicate with the business logic layer through the presentation layer, while the data layer is separated. This greatly improves the security and consistency of enterprise data, and at the same time, due to the layered design, it also improves the efficiency between each layer, making it more conducive to project development, updates, and maintenance.

傳統使用一個台PC操作所有的系統

三層架構將服務分層，使用者可透過客戶端同時進行系統操作並保持安全性

System Expansion
For various considerations and to maintain consistency and security, application architectures also change accordingly. But what if the system faces resource bottlenecks?
Here, two methods of system expansion will be used to achieve a highly available system architecture.

Vertical Scaling
When the resources used by the system face bottlenecks, you can upgrade the hardware specifications of a single host to solve the load problem, including CPU, Memory, GPU, hard disk, network card, etc.

Horizontal Scaling
In vertical scaling, some problems that vertical scaling cannot solve will ultimately be encountered. For example, each expansion update requires a downtime operation, and the continuous upgrading of system hardware also faces hardware limitations such as OS CPU, hard disk, and network card throughput. The cost-effectiveness of vertical scaling may be lower than horizontal scaling, which involves expanding through multiple hosts.

The main difference between horizontal scaling and vertical scaling is that horizontal scaling does not increase the hardware specifications of a single host to cope with service load issues. Horizontal scaling increases the number of hosts and uses load balancing to distribute the overall service load.

Server Load Balance
Load balancing devices accurately distribute requests through various algorithms and play a crucial role in the entire horizontal expansion.

Brief Introduction to Several Common SLB Algorithms:

• Round Robin: Each request is distributed to servers in turn, from the first to the Nth, and the process repeats, which is currently the most commonly used balancing method.
• Weighted Round Robin: The load ratio is set according to the different performance of servers. Increase the load on high-performance servers and decrease the load on low-performance servers.
• Random: Requests are randomly assigned to servers, without following any order.
• Weighted Random: The load balancing device continues to send ICMP requests to the server and allocates requests to the server with the fastest response based on the returned data.
• Least Connections: Records the number of connections established to each server and allocates the connection to the server with the least number of connections each time a new request is made.

2.How Cloud Solves System Bottlenecks

Since 2010, cloud applications have gradually doubled in growth every year, gradually replacing the traditional model of enterprises building their own data centers. Cloud computing is not only very friendly to cost control and emerging enterprises but also allows flexible rental services without large upfront costs, including various tools, making it more convenient for enterprises in development and architectural deployment.

AWS(Amazon Web Services)
Regarding how Amazon rose to prominence, this article does not elaborate. Amazon pioneered a new choice of information services, including convenience, diverse resources, a pay-as-you-go business model, and no restrictions on users. These novel advantages enable more emerging industries to create infinite possibilities on the shoulders of this giant.

雲上解決方案:以目前雲端領導服務商AWS為例

▲ 根據Gartner於2020全球市佔率（數據來源：Gartner（June 2021））

Cloud computing is like a puzzle that needs to be envisioned in the mind and then assembled using the tools available in the cloud toolbox. Below, we will introduce how to use cloud components to solve the problems arising from high concurrency in the cloud.

Elastic Compute Cloud(EC2)
A virtual server provided by Amazon, offering various specifications to choose from, allowing for the easy and rapid provision of complete host resources.

Elastic Load Balancing(ELB)
As a crucial component of high-concurrency architecture, the load balancer provides load distribution, handling concurrency issues while also providing overall service fault tolerance, significantly increasing availability.

• Application Load Balancer (ALB): Optimizes and adjusts load balancing for the OSI layer 7 - application layer, suitable for handling HTTP/HTTPS requests.
• Network Load Balancer (NLB): Optimizes and adjusts load balancing for the OSI layer 4 - transport layer, suitable for handling TCP and UDP requests.

Autoscaling
Provides automatic scaling and resource utilization functions, adjusting virtual host resources based on different percentage rates and limiting the number of host resources.

Autoscaling can flexibly scale based on the following percentages:

• Average CPU usage
• Average incoming network traffic
• Average outgoing network traffic
• Average ALB target request count

RDS services in the cloud are like installing database software on your own server, but the difference is that the cloud service provider manages the RDS system-level control, management, and security, allowing users to focus solely on database application design. RDS also provides different availability options.

• Multi-AZ Deployment: Synchronous standby databases can be established in different availability zones, automatically transferring when the primary database is abnormal or undergoing maintenance.
• Automatic Storage Expansion: You can set storage space thresholds, automatically extending storage space when storage capacity exceeds the threshold.
• Serverless Database Applications: Designed for unpredictable database loads, allowing the database to be turned off when not in use to save costs (Aurora).
• Elastic Concurrency Database: Can choose between single write-multiple read replicas or multiple write replicas depending on the usage scenario (Aurora).

CloudFront
CDNs have gradually become necessary considerations in global network services. The advantage of the cloud's global distribution is the ability to deploy rapidly and utilize caching mechanisms, allowing visitors to connect to the nearest node to access services, providing a good service experience.

ElastiCache for Redis
A cache database in the cloud, Redis utilizes memory as storage media, horizontal scaling, and the Key-Value feature to significantly increase data read and write speeds several times over, especially during heavy database read and write operations.

Simple Storage Service(S3)
S3 provides a web service interface for easy data access storage services, with high availability, durability, and excellent scalability. It can also be used to host static web pages with its powerful capabilities.

With a basic understanding of the above AWS products, the following will integrate these puzzle pieces to export the final high-concurrency architecture in the cloud.

1. Differentiate between two availability zones A and B to prevent a single availability zone from experiencing disruptions.
2. Utilize load balancing (ELB) in conjunction with auto-scaling (Auto Scaling) to automatically scale servers based on web loads.
3. Implement a master/backup mode for the database (RDS) and establish backups in S3 storage through synchronous redundant databases.
4. Use a caching database (Redis) to cache RDS and establish synchronous backups in availability zone B.      a. When EC2 applications read from Redis, if there's a cache hit, Redis quickly returns data to EC2.      b. When EC2 applications read from Redis, if there's a cache miss, query RDS and write the result to Redis for quick retrieval next time.
5. Store static files in S3 to distribute website loads, such as images, videos, and other content.
6. Use CDN (CloudFront) to cache and distribute load-balanced and S3 content, allowing visitors to connect to the nearest node to access website content.