However, this limits the firing rate of the weapon, because opening and closing the chamber needs to be done at a certain speed so that the old case can eject out of the port before the chamber closes. This means that the bolt return spring has to be designed and manufactured so that this happens correctly. It was reasoned then that if the case could be eliminated from the cartridge, there would be no need to go through this ejection cycle.
We've actually seen a weapon already where there is no need to pull out the old casing after the weapon is fired. Matter of fact, it was invented in the early 1820s and perfected in 1836 and was one of the first widely-used breech-loading military rifles. We're talking about the Dreyse Needle Gun and its Needle Gun Cartridge. The needle gun cartridge was made of paper, but it was also coated with a greasy substance that provided lubrication to the weapon. When the gun was fired, the gunpowder would burn and shoot the bullet out of the tube. Since the primer was attached to the back of the bullet, the priming cap would also leave the gun with the bullet. The burning gunpowder would also burn up the cartridge case, so when the user opened the breech to load another bullet, often there was very little ash left behind to clear and nothing else.
An attempt was made in 1846 to provide a caseless cartridge by making a very hollow bullet and putting the gunpowder in that hollow. This was called the Rocket Ball. The Volcanic Repeating Arms company modified this idea by putting a primer cap at the base of the bullet. These ideas were not very successful though, as the main drawback was that the length of the bullet limited the amount of gunpowder that could be used. This meant the firearms using this ammunition were extremely underpowered.
The Germans attempted to make caseless ammunition during WW-II, so that they could avoid using metal (which was in short supply during the war) for their cartridge cases. However their attempts during the war were not very successful.
A German company did successfully invent a caseless cartridge weapon in the 1970s though. Messrs. Heckler and Koch of Germany developed a military rifle, the G11, which uses caseless ammunition. The cartridge is a stick of nitrocellulose with a bullet glued on one end, and a primer glued on the other end. The G11 can provide a much higher rate of fire due to the fact that it doesn't need to eject any spent cartridges. Hence it can fire at over 2000 rounds/minute, when a conventional automatic assault rifle like the AK-47 tops out at 650 rounds/minute. The G11 uses a complicated mechanism to feed the ammunition at this rate. Unusually for cartridges, these are square in cross-section. The link here shows a picture of a caseless G11 cartridge (courtesy of wikipedia.com)
One of the early problems encountered by the G11 was the tendency of the ammunition to "cook-off". One of the secondary features of a metallic cartridge case is that it acts as a heat-sink, i.e. it draws off some of the heat from the burning gunpowder and keeps the chamber from getting extra hot. Also when a new cartridge is inserted into the hot chamber, the metallic case keeps the gunpowder inside the cartridge relatively cool so that it does not ignite automatically from the heat. Without a cartridge case around, the G11 chamber gets very hot after a few rounds have been fired. Due to this, it had a tendency for the remaining cartridges to automatically fire without pulling the trigger, since they were getting fired by the heat in the chamber. This is the reason why the G11 was pulled off the NATO trials of 1979. To solve this, Heckler & Koch had to team up with Dynamit Nobel of Sweden to produce a new propellant instead of nitrocellulose. This new propellant had a much higher ignition temperature.
A secondary problem was how to pull out an unfired/misfired cartridge. With a metallic case around a cartridge, it is pretty easy to pull the cartridge out by its raised rim. With a caseless cartridge, this is much harder and thus the firing mechanism needs to be designed to take this into account. As a result, the G11 firing and feed mechanism is much more complicated.
However, the G11 never reached much popularity because of the earlier cook-off tendencies and the fact that the firing mechanism was very complicated and made the weapon very expensive. Hence it was replaced by the lower cost H&K G-36 weapon which was much cheaper to manufacture and used a conventional centerfire cartridge.
Another approach to solving the cook-off problem was pioneered by the Austrian company Voere. In their Voere VEC-91 rifle, instead of a hammer or a firing pin striking the primer, they use an electrical system to ignite the primer. Power is provided via two 15-volt batteries in the weapon's grip and allow it to fire upto 5000 rounds before replacing the batteries. An Australian company called Metal Storm also produces weapons that use caseless ammunition and electrical ignition. Neither of these has had much commercial success, however research does continue into caseless cartridge technology.