Not a problem
First, I assume that by 'no animals' you mean animals big enough to see. If you remove the micro-organisms, then there is a big problem. But as long as you allow the various soil bacteria and nematodes and such, then no big deal.
Some people say pollination will be a problem. This is completely wrong: let me explain. First off, the biggest staple crops in the world (rice, beans, wheat, maize, soybeans) are wind pollinated. Second, root crops and leaf crops do not need to be pollinated to produce the parts that we eat; and many of them do not need pollination do produce seed (though that is nice for genetic diversity). Third, many of the domesticated hybrids are self-pollinating. This was a crucial step in domestication, and it is not a modern phenomenon. Triticum durum or Durum Wheat (used for couscous and pasta) is the first known agricultural hybrid and is first attested about 7000 BC. So not only do we have self-pollinating hybrids now, but producing them is not super hard. And fourth, many tree-like domesticates are propogated by cutting or grafting; such as olives and bananas.
Another objection is that some people say worms are important. This is again not true; worms are important for mixing up the soil layers. This is something that can be done with a hoe, if needed. The actual breaking down of organic matter in the soil is done by microbes (in the worm's gut) which can happen just as well in the soil...or a compost heap.
What to use
For tool, stone and bone work great. There aren't a lot of animals with bones, but humans have plenty. As long as there are shrubs, you can make some smaller wooden tools as well.
For clothing, cotton and linen (from flax) work fine, depending on the climate. Cold climates will be bummer without wool and hides, but you can just layer on the linen.
For food, you have grains, pulses, oilseeds and leafy vegetables aplenty.
For fuel, you still have bushes to burn, and there are plenty of coal mines around too.