In the days prior to industrialized farming it took about 1 acre of land per person being intensively worked constantly to produce a sustenance level yield. This is actually why families used to be huge (and still are in the developing world.) There is no way without petro-chemical based pesticides, herbicides, and fertilizers that we could ever support our current population. Prior to the use of herbicides one could expect to lose from 1/4th to 1/3rd of the crop before harvest just due to pests and weeds, and then could then very easily lose that again following the harvest due to further pilferage of the stores from pests. So on a good year with a bumper crop you are losing up to half of your yield before you can even eat or sell it.
Follow this with industrial equipment for farming. The introduction of even the first rudimentary steam tractors explosively improved yield because now one person could easily till, plant, and harvest up to 100 acres each. Add in even the early fertilizers, pesticides, and herbicides and suddenly one family is easily producing enough food for 1,000 people on only a few hundred acres. People really have no idea how truly revolutionary industrialization was for farming. Prior to the advent of petro-chemicals particularly famines were pretty much unavoidable cyclical events.
I'm sure various energy solutions could replace the diesel driven tractors, but in the event of a true and sudden petro-crisis our agriculture output would plummet due to not having these chemicals available. Until a suitable alternative could be found (which can take years since chemistry is a bit different than just redesigning tractors to run off of electric) you would see famines on a biblical scale. Whats worse, thanks to the unrest and economic events that typically surround famines, it could easily become self-sustaining since the social disruption could delay or halt altogether the development of new petro-alternative agri-chemicals.
Alternative Energy Solutions Wouldn't Replace Lost Land
As we sit present arable and productive farmland only makes up about 1% of available land. A very very large amount of the land we use for farming was not suitable for farming until petro-chemicals came into use. So we wouldn't just have lower yields and more crop loss without agri-chemicals, we would have less useful land as well. Famine would set in almost immediately.
Famines Cause Plagues, Wars, Prevents Progress
So now that everyone is hungry they start getting sick, in addition to weakened immune systems from malnutrition, you have the medicine shortages mentioned earlier. In addition to this we now need about an acre per person to produce food and we are losing a lot of that too, so wars over arable land would become commonplace. Societies would begin desperately trying to support as many of their starving masses as possible. Developing new alternatives to petroleum based agri-chemicals could take a decade plus the time to distribute and implement them. But your people are hungry NOW, and when humans get hungry and desperate they tend to fight. Since we live in organized societies this usually means large scale wars. You could easily see a "vicious cycle" develop where remaining resources that could be allocated for research get gobbled up by the military instead. If the good land is being fought over its not being used for growing things, so more people die, the military needs to seize more land, gobbling up more resources etc etc. It wouldn't take too many rounds of this cycle to kill off enough of the population and burn off enough resources that an actually effective response becomes impossible.
In the event of a sudden loss of petroleum based agri-chemicals you would be too hungry to notice not having cheap plastics. This means food is the first and most important material you would lose access to.