This is indistinguishable from Earth's gravity
Centrifugal acceleration on an object is $$\mathbf{a} = \left[\frac{d^2\mathbf{r}}{dt^2}\right] + \frac{d\omega}{dt}\times\mathbf{r}+2\omega\times\left[\frac{d\mathbf{r}}{dt}\right]+\omega\times\left(\omega\times\mathbf{r}\right). $$
If we assume that the particles of interest are at a contant distance from the star ($\mathbf{r}$), then the first and third terms are set to zero. If we assume that rotational velocity is contant, then the second term is set to zero. That leaves \begin{equation}\mathbf{a} = \omega\times\left(\omega\times\mathbf{r}\right). \end{equation}
In order to feel a force equivalent to Earth's surface gravity ($g = 9.8$ m/s$^2$), we solve
$$9.81 = \omega\times\left(\omega\times1 \text{ AU}\right).$$
Now, this is highly dependent on the polar angle of the location of interest. The above equation only holds at the equator. At other angles, the centripetal acceleration will be less! Let us assume that you want 1$g$ at the equator, and less towards the poles. Since $1 \text{AU} = 1.496\times10^{11}\text{ m}$, we can solve for $\omega = 8.1\times10^{-6}$ radians/sec to achieve equatorial centripetal force equivalent to Earth's gravity. This means the Dyson sphere has to rotate around the sun every 775,909 seconds; about 9 days. That is pretty fast, but since this is a Dyson sphere, I assume the material structure can handle it.
Now, to answer your specific question, what will the centripetal force be like above the surface. Lets say that we are 10 km above the equator; that is higher than Mount Everest. This affects $\mathbf{r}$; the distance from the sun. The distance from the sun of the surface is $1.496\times10^{11}\text{ m}$. Subtracting 10,000 meters from that we get....$1.496\times10^{11}\text{ m}$. Compared to 1 AU, any altitude we might be used to on Earth is insignificant.
Don't forget the poles!
As an additional aside; please remember that areas off of the equator will not have the same 'gravity' as the equator. Using the equation
$$\mathbf{a} = \omega\times\left(\omega\times\mathbf{r}\right),$$
let us assume that we are at a polar angle of $\phi$ in radians; that is $\phi$ radians from the pole and $\pi/2 - \phi$ radians from the equator.
The cross product, featured above, can be represented by
$$ a \times b = ||a||\,||b||\sin{\theta}$$
where $\theta$ is the angle between vectors. Conveniently, the polar angle ($\phi$) is the angle between $\omega$, which is the axis of rotation, and whatever point on the surface of the Dyson sphere.
When we evaluate the inner portion of
$$\mathbf{a} = \omega\times\left(\omega\times\mathbf{r}\right),$$
the angles $\theta = \phi$. The vector result of the inner portion is perpendicular to both of the operands. Therefore, when evaluating the outer portion, we simply multiply $\omega$ by the inner result. Therefore, we can say that
$$ a = ||\omega||^2\, ||r||\sin{\phi}$$.
Lets plug in the equivalent of the Tropics on Earth, 23$^\circ$ latitude; 67$^\circ$ or 1.17 radians from the pole. With the already established rotational velocity:
\begin{align} a &= ||\omega||^2\, ||r||\sin{\phi}\\ &=g\cdot\sin{\left(1.17 \text{ rad}\right)}\\ &=0.92\cdot g \end{align}
The effective force of gravity falls to 92% at 23$^\circ$ from the equator. It falls to 77% at 40 $^\circ$ latitude; and down to 50% at 60 $^\circ$ latitude.
Conclusion
Traveling up into the air above a Dyson sphere designed to simulate gravity with centrifugal force at the equator does not affect gravity appreciably. Traveling towards the poles, on the other hand, does.