Something that is moving upstream is moving towards the source of a river, from a point further down the river. Something that is upstream is towards the source of a river.
e.g. The water rose high enough for them to continue upstream.
e.g. ...the river police, whose headquarters are just upstream of the Isle St Louis...