I think a big question that hasn’t been brought up in the previous answers is how are these people going to deal with bacteria, parasites, and interactions with other native wildlife like large predators, venomous snakes, and stinging insects. If only living organic matter can pass through the portal that means the people coming through will have no clothes, no footwear, no camping supplies, and most important no medicine. Antibiotics have to be carried in containers, after all, and may not be able to pass through since they aren't living matter.

Viruses may not be that big of an issue because many viruses are thought to have transferred to humanity from domestic animals, and if you’re in South America the only easy source of viruses would be New World monkeys (compare Africa, where you might get viruses from apes or even non-human hominins depending on the setting). The big issue will be parasites, and in pre-urbanized societies parasites are a much larger issue than viruses. Some examples known from South America can be found here. The big one? Malaria, which would be worse without any anti-malarial drugs you can bring with you. There are a lot of parasites in water bodies around the world, especially in tropical climates, and others can be transmitted to humans if they eat the local wildlife (our tapeworms originally came from a species that infests lions and hyenas, for example). Parasites reduce the quality of life for the settlers, may require other individuals to care for them, and can cause disability or even death, especially in a world without anti-parasite drugs.

The Amazon is about the worst place in the world to try and settle without the aid of a pre-existing technological base, as its high biodiversity and warm, wet climate means that parasites and stinging insects are everywhere, compared to somewhere like a grassland or desert (and it's actually been suggested this is one reason why human civilization first developed away from heavily forested areas).

Additionally, the nature of the portal means no guns or even spears and bows and arrows to defend against attacks from jaguars and caimans. At best weaponry has to be made from scratch. If this is a world where humans never got to South America it also means Smilodon, dire wolves, short-faced bears, and Protocyon troglodytes, all of which have been found in Amazonian Brazil and are known to eat large (read: human-sized) mammals, would still be a major issue, as these species are thought to have been wiped out by humanity (e.g., humans competing with them for food). Exactly how common large predators are without humans to reduce their numbers by hunting them and competing with them for food is often greatly underappreciated (for example, 20,000 years ago carnivorous mammals were more common on the humanless North American prairies than on the human-filled African Serengeti). Without human interaction these animals would have no fear of humans and would see them as new and potentially interesting food (like polar bears in the Arctic). Something an absence of weapons would do nothing to dissuade.

How this relates to the larger question of how long it would take to rebuild modern society is that constantly having to spend time caring for sick or injured individuals or send them back through the portal for treatment, losing people who could be working due to disease, having to constantly deal with predators (or even something like a big, aggressive gomphothere or ground sloth that wanders into camp), and replacing manpower losses from predators and the associated psychological effects (think Tsavo) would slow rebuilding society to a crawl until a proper technological base is built up. Given you are trying to do it in a region that is not amenable to traditional agriculture, has lots of parasites, few exposures of rock to mine iron and stone, lots of small biting wildlife, extreme floods, and dangerous megafaunal predators it might take centuries. On the other hand, surviving in this world and actually trying to build this technological base could make good grist for your story.

I think a big question that hasn’t been brought up in the previous answers is how are these people going to deal with bacteria, parasites, and interactions with other native wildlife like large predators, venomous snakes, and stinging insects. If only living organic matter can pass through the portal that means the people coming through will have no clothes, no footwear, no camping supplies, and most important no medicine. Antibiotics have to be carried in containers, after all, and may not be able to pass through since they aren't living matter.

Disease will cause massive casualty rates without access to medicine. Viruses may not be that big of an issue because many viruses are thought to have transferred to humanity from domestic animals, and if you’re in South America the only easy source of viruses would be New World monkeys (compare Africa, where you might get viruses from apes or even non-human hominins depending on the setting). The big issue will be parasites, and in pre-urbanized societies parasites are a much larger issue than viruses. Some examples known from South America can be found here. The big one? Malaria, which would be worse without any anti-malarial drugs you can bring with you. There are a lot of parasites in water bodies around the world, especially in tropical climates, and others can be transmitted to humans if they eat the local wildlife (our tapeworms originally came from a species that infests lions and hyenas, for example). Parasites reduce the quality of life for the settlers, may require other individuals to care for them, and can cause disability or even death, especially in a world without anti-parasite drugs.

The Amazon is about the worst place in the world to try and settle without the aid of a pre-existing technological base, as its high biodiversity and warm, wet climate means that parasites and stinging insects are everywhere, compared to somewhere like a grassland or desert (and it's actually been suggested this is one reason why human civilization first developed away from heavily forested areas). At least in a desert or grassland you don't have to constantly worry about parasites and disease-carrying insects.

Food is also going to be a major problem because as per OP you can't bring dead organic matter through the portal. So all food has to be gathered on site and you likely can't gather enough in the local area to feed a large group of people without tools to increase efficiency. At the least you might strip the area bare of edible plants. Catching game is going to be difficult without tools or tools with which to build tools (scrapers, appropriate striking tools and material to make arrowheads, sinew or sap to attach arrowheads onto shafts).

And while you can bring livestock through to kill for meat and leather, you will have to bring them through alive and have no tools with which to kill and butcher them. And when you do you have no way of keeping them from wandering off or trampling you in a panic when you try to kill them because all halters, leadropes, and reins will disappear the minute you go through the portal. It is hard to restrain a horse, cow, or other large livestock without a halter, because even docile livestock tend to do what they want when not restrained and given how big they are it is difficult to stop them.

Apex predators would be a huge problem. The nature of the portal means no guns or even spears and bows and arrows to defend against attacks from jaguars and caimans. At best weaponry has to be made from scratch. If this is a world where humans never got to South America it also means Smilodon, dire wolves, short-faced bears, and Protocyon troglodytes, all of which have been found in Amazonian Brazil and are known to eat large (read: human-sized) mammals, would still be a major issue, as these species are thought to have been wiped out by humanity (e.g., humans competing with them for food). Nothing short of a firearm or atlatl is going to stop them.

Exactly how common large predators are without humans to reduce their numbers by hunting them and competing with them for food is often greatly underappreciated (for example, 20,000 years ago carnivorous mammals were more common on the humanless North American prairies than on the human-filled African Serengeti). People often forget that the world we live in has been massively shaped by humans even before the invention of agriculture, and that without humans the world would likely be swarming with large predators and megafauna. Without human interaction these animals would have no fear of humans and would see them as new and potentially interesting food (like polar bears in the Arctic). Something an absence of weapons would do nothing to dissuade.

How this relates to the larger question of how long it would take to rebuild modern society is that constantly having to spend time caring for sick or injured individuals or send them back through the portal for treatment, losing people who could be working due to disease, having to constantly deal with predators (or even something like a big, aggressive gomphothere or ground sloth that wanders into camp), and replacing manpower losses from predators and the associated psychological effects (think Tsavo) would slow rebuilding society to a crawl until a proper technological base is built up. 

Given in this scenario people are trying to rebuild civilization in a region that is not amenable to traditional agriculture, has lots of parasites, few exposures of rock to mine iron and stone, lots of small biting wildlife, extreme floods, and dangerous megafaunal predators it might take centuries. It's possible it may not ever happen or if it succeeds it succeeds due to sheer luck, sending people through the portal likely resulting in massive casualty rates. Sending people through and trying to rebuild civilization without medicine, food, tools, or means of protection sounds like either a desperation move or a particularly cruel form of execution.

Even in cases like Discovery Channel's Naked and Afraid many of the contestants have to abandon the challenge due to injury or illness or be sent to the hospital shortly thereafter, and even those that complete the challenge are usually suffering from starvation and dehydration by the end. Humankind's biggest weakness is that without tools we are pretty much helpless or at least on the defensive until we can get tools, and we often rely on already-existing tools to make more tools. There's a reason why in most of these survival shows the goal is "get back to civilization" rather than the more ambitious "rebuild civilization". On the other hand, surviving in this world and actually trying to build this technological base could make good grist for your story.

I think a big question that hasn’t been brought up in the previous answers is how are these people going to deal with bacteria, parasites, and interactions with other native wildlife like large predators, venomous snakes, and stinging insects. If only living organic matter can pass through the portal that means the people coming through will have no clothes, no footwear, no camping supplies, and most important no medicine. Antibiotics have to be carried in containers, after all, and may not be able to pass through since they aren't living matter.

How this relates to the larger question is constantly having to spend time caring for sick or injured individuals or send them back through the portal for treatment, losing people who could be working due to disease, having to constantly deal with predators (or even something like a big, aggressive gomphothere or ground sloth that wanders into camp), and replacing manpower losses from predators and the associated psychological effects (think Tsavo) would mean rebuilding modern society would slow to a crawl until a proper technological base is built up. Given you are trying to do it in a region that is not amenable to traditional agriculture, has lots of parasites, little exposures of rock to mine iron and stone, lots of small biting wildlife, extreme floods, and dangerous megafaunal predators it might take centuries. On the other hand, surviving in this world and actually trying to build this technological base could make good grist for your story.

Parasites, Bacteria, Biting Wildlife, and Apex Predators Would Make It Take A Really Long Time

I think a big question that hasn’t been brought up in the previous answers is how are these people going to deal with bacteria, parasites, and interactions with other native wildlife like large predators, venomous snakes, and stinging insects. If only living organic matter can pass through the portal that means the people coming through will have no clothes, no footwear, no camping supplies, and most important no medicine. Antibiotics have to be carried in containers, after all, and may not be able to pass through since they aren't living matter.

How this relates to the larger question of how long it would take to rebuild modern society is that constantly having to spend time caring for sick or injured individuals or send them back through the portal for treatment, losing people who could be working due to disease, having to constantly deal with predators (or even something like a big, aggressive gomphothere or ground sloth that wanders into camp), and replacing manpower losses from predators and the associated psychological effects (think Tsavo) would slow rebuilding society to a crawl until a proper technological base is built up. Given you are trying to do it in a region that is not amenable to traditional agriculture, has lots of parasites, few exposures of rock to mine iron and stone, lots of small biting wildlife, extreme floods, and dangerous megafaunal predators it might take centuries. On the other hand, surviving in this world and actually trying to build this technological base could make good grist for your story.