Wooden ships. You can build on the hulks of wooden ships. There are a lot of ships under San Francisco as shown in this map.

from http://www.upout.com/blog/san-francisco-3/map-shows-ships-buried-underneath-san-francisco-2

It made news when they discovered an old sailing ship under the ruins of the World Trade Center. Wooden boats were apparently used for landfill not infrequently.

from http://www.huffingtonpost.com/2014/07/30/world-trade-center-ship-mystery_n_5634280.html

There is nothing wrong with your swamp city built on heaps of wood. Cut down the forest, pile it up and build on it. It would be more interesting to have it be built on an abandoned fleet of some sort. Wood rots but buried wood can last a long time in the anaerobic environment.

Ceramic Your people don't have stone, but they have mud and they have wood. They can make bricks. They can make ceramics. The Romans made both in abundance.

https://en.wikipedia.org/wiki/Monte_Testaccio Monte Testaccio is a large hill built entirely of discarded Roman ceramic jugs.

It has a circumference of nearly a kilometre (0.6 mi) and stands 35 metres (115 ft) high, though it was probably considerably higher in ancient times

That is pretty cool. A dedicated brickworks and a giant stack of bricks should be doable in your swamp. I learned about [tells](https://en.wikipedia.org/wiki/Tell_(archaeology) on the worldbuilding stack a few weeks ago. They are big. They are mostly old mud brick.

Wooden ships. You can build on the hulks of wooden ships. There are a lot of ships under San Francisco as shown in this map.

from http://www.upout.com/blog/san-francisco-3/map-shows-ships-buried-underneath-san-francisco-2

It made news when they discovered an old sailing ship under the ruins of the World Trade Center. Wooden boats were apparently used for landfill not infrequently.

from http://www.huffingtonpost.com/2014/07/30/world-trade-center-ship-mystery_n_5634280.html

There is nothing wrong with your swamp city built on heaps of wood. Cut down the forest, pile it up and build on it. It would be more interesting to have it be built on an abandoned fleet of some sort. Wood rots but buried wood can last a long time in the anaerobic environment.

Ceramic Your people don't have stone, but they have mud and they have wood. They can make bricks. They can make ceramics. The Romans made both in abundance.

https://en.wikipedia.org/wiki/Monte_Testaccio Monte Testaccio is a large hill built entirely of discarded Roman ceramic jugs.

It has a circumference of nearly a kilometer (0.6 mi) and stands 35 meters (115 ft) high, though it was probably considerably higher in ancient times

That is pretty cool. A dedicated brickworks and a giant stack of bricks should be doable in your swamp. I learned about tells on the worldbuilding stack a few weeks ago. They are big. They are mostly old mud bricks.