Normally when the computer is switched on and the CPU first receives power it will read a specific memory location (fixed by the manufacturer) to get the address where the first instruction of the first program it must execute is stored.
Both the memory location and the first program are stored in ROM (Read Only Memory) and so instead of being called "Software" it is often called "Firmware". This first program is normally called the BIOS (Basic Input Output System) and it runs continuously when the computer is on. It deals with input key presses on some keyboard and basic output to some kind of display. It also deals with the mouse and hard disc if there is one and perhaps a few other devices. Some expansion cards have "Option ROMS" which contain code that needs to be run for them to operate properly. The BOIS can also find and run this code.
It then normally reads the hard disc boot sector (if there is a hard disc) to copy into RAM (or load) and execute another program, the boot loader, responsible for loading and executing the Operating System which is also on the hard disc, be that Linux, MacOS or Windows. The Operating system consists of a number of programs which not only provide the user interface we see but much else besides.
You can find out more about the BIOS from: Wikipedia.