When Rudolph diesel first set out to build a more efficient engine, he needed to start with compression ignition. A normal spark ignition engine compresses the fuel air mixture then when it’s fully compressed it fires a spark across the spark plug. The spark ignites the mixture and moves the piston turning the engine. The compression of a spark ignition engine is limited by the octane number. If the compression is too high the engine will pre-ignite which is a condition that will destroy the pistons and the rest of the engine. More power and higher efficiency is possible at higher compression levels. That is the reason racers use race gas which is simply higher octane fuel. The race gas doesn’t provide more power (unless oxygenated) the increased compression does.
The diesel engine has much higher compression ratio around 20:1 versus the 10:1 for a spark ignited engine. Diesel fuel has a much lower octane number which you would think would make thing worse. Unlike a conventional engine the piston travels upward and compresses the air without fuel to it’s maximum amount. This way the fuel can’t pre-ignite and harm the engine. The diesel engine waits until the mixture is compressed and sprays the required amount of fuel into the chamber. The direct injection is very high pressure to overcome the compression ratio in the cylinder. Diesels use mechanical injection for the fuel injection. Similar direct injection usually used on 2 stroke outboard engines use air assist or only electronic pump fuel pressure to achieve the much lower pressures needed there. The diesel fuel makes mechanical injection an option because it’s not dry like gas and actually lubricates the pump. As the diesel fuel gets sprayed into the chamber it self ignites due to the very high temperatures in the chamber. Air temperature gets very high due to the air being compressed so high to the 20:1 ratio. This is the reason that a lower octane rating fuel is used to allow the fuel to burn easier. The diesel fuel is control able unlike gasoline which pre-ignites and causes damage to the engine. The timing on a diesel is actually controlled by the injection timing. Between the controlled injection and the much heavier duty engine parts, diesels are very reliable. Most diesels can’t spin very high rpm’s. This is due in part to the high compression ratio, as the engine has a harder time turning against it. The high ratio makes the engine more efficient because more compressed air equals more oxygen in the cylinder. The additional oxygen aids in burning the fuel better, meaning more a efficient engine.