Kevin Feige, president of Marvel, has told D23 Magazine that sequels for "Thor" and "Captain America: The First Avenger" are already in the development stages. Both sequels will hit the theaters after "The Avengers" is released.
Feige said after both team up in "The Avengers", "Thor will go off into a new adventure, and Captain America will continue to explore the modern world in another film of his own. We hope that holds true for the characters appearing in that film - Black Widow, Hawkeye, and the spy organization, S.H.I.E.L.D. - all of whom are more than worthy and capable of carrying their own films."
He also stated that there are "a lot of other characters we're prepping and getting ready for film debuts: the world of martial arts, these great cosmic space fantasies, Doctor Strange, and the magic side of the Marvel Universe. There are many, many stories to be mined."