Yes and No
The problem, as it stands, is due to two factors:
- In order to make the system strong enough to perform the functions, even with the use of exotic materials, it has to have a minimum weight.
- In order to shift that minimum weight, you need more power than can realistically be generated.
Adding more fuel adds more weight, adding a bigger engine adds more weight etc.
Now - there are 'jet packs' that have been designed and tested and flown, but they are of limited use because of the above problem.
To answer your world building question - if Power Source is not an issue, then it doesn't really matter what material you use, you could create a concrete aircraft with a sufficiently powerful engine (Not that I would recommend that...)
If materials is not an issue, then any current engine and power source should be sufficient.
The challenge with modern day tech is that for the weight of materials we need to make it safe and practical, we don't have a power source that can provide the output for long enough to make it a realistic travel option.