The parallel processing of computers includes pipeline, super pipeline, super scalar, multi-function unit, array processing and multi-computer processing technologies. Pipeline, super-pipeline, super-scalar and multi-function unit technologies have been widely used in various computers. Therefore, parallel processing systems mainly refer to multi-processors and multi-function component systems. In the 1960s, the parallel pipeline and multi-function unit technology between instructions developed rapidly and was widely used. In the mid-1970s, pipeline-based vector machines that can perform parallel processing on ordered data sets and parallel systems composed of multiple vector machines became the mainstream. This model is a single instruction stream multiple data stream (SIMD) computer. After the 1990s, multi-computer systems with parallel programs, tasks, or processes developed rapidly and became the mainstream of parallel processing systems. This model is a multi-instruction-stream and multi-data-stream (MIMD) computer. There are three main types of computers with multiple instruction streams and multiple data streams: ① Shared memory multiprocessor system (SMP). All processors in the system share a memory through the Internet, and there is a unified memory address space, and each processor communicates through shared variables. In order to improve the scalability of the system, the shared memory can also be distributed in each processor, which is called the Scalable Shared Memory Multiprocessor System (SSMP). ②Message transfer multiprocessor system. Each processor in the system has its own local memory, and each processor is connected by the Internet to form a complete system. Each processor in the system runs its own operating system and communicates through message passing. Among them, a parallel processing system that uses an Internet to connect multiple workstations or microcomputers is also called a cluster system. ③Hybrid multiprocessor system. A local memory is first shared by several processors to form a node, and then a computer system is formed by multiple such nodes through the Internet, and the nodes communicate with each other by means of message transfer. In addition, a parallel processing system composed of many vector machines is called a multi-vector machine processing system. Single instruction stream and multiple data stream computers mainly include vector machines and array processors. The array processor is a parallel processing system in which a controller interprets instructions and controls multiple processing components with the same function to perform the same operations on their data. The key technology to develop a multi-computer parallel processing system is to realize communication and synchronization between the processors. Commonly used interconnection networks such as buses, crossbar switches, grid networks, hypercubes, fat trees, and multi-level Internet realize inter-machine communication. Large-scale multi-machine systems often use hardware to support processor synchronization. Parallel operating systems are generally distributed on each processor, and are responsible for scheduling each processor to work in full and coordinated manner. Parallel languages are generally developed or expanded from serial languages. Parallel language compilers must give full play to the characteristics of parallel processing systems with different structures to achieve high efficiency. In addition, parallel programming environments, parallel debugging environments, and parallel algorithms are also very important. Small-scale parallel processing multi-machine systems will become more popular due to high cost performance. Massively parallel processing systems will develop towards the goals of low power consumption, low footprint and low prices while improving performance. The development of chips containing multi-processors and multi-threads will be a good way to realize this system. The massively parallel vector machine system has fast node processing speed, high memory access bandwidth, and can obtain higher system performance, and has shown good development prospects. Quantum computing, quantum devices, superconducting switches, and optical switches are promising technologies in the future. In particular, quantum computers composed of quantum devices will provide a larger-scale parallel computing environment and become a very important development direction for parallel computing systems.
