The East Anglian Bangladeshi Islamic Trust received permission from Norwich City Council this month to build the facility on Aylsham Road.
As well as a mosque, the scheme will see an old building turned into a community cafe.
Management committee secretary Sirajul Islam said the new building was needed to accommodate the East Anglian Islamic Centre’s growing congregation of 250 people, which is currently based on Rose Lane.
Mr Islam said: “It has been a long-term aspiration for our community to have a building of our own and to have an identity in the city.
“We have been looking for a bigger form of accommodation because our community is slowly growing and Rose Lane does not have enough space to accommodate the congregation.”
Mr Islam stressed that the old building, which was built in 1902, will not be demolished, the Eastern Daily Press reported.
Instead, he said it will be refurbished and turned into a cafe for people to use regardless of their religious background.
He said people living around the proposed site had been “very welcoming” of the plans.
The mosque will include a prayer hall, imam room, kitchen and toilets on the first floor.
Meanwhile the ground floor will be made up of a counselling room, a gymnasium, prayer room, funeral room and office.
Mr Islam said the land, which includes the pub site and large garden to the rear of the property, cost around £450,000 to purchase.
The trust now needs between £800,000 to £1m to build the new mosque and refurbish the old building.
It has three years to raise the money and begin building work, Mr Islam said.
The site, which is between Waterloo Park Avenue and Edmund Bacon Court, will have 20 parking spaces.
Mr Islam said a further 50 car park spaces will be available at the nearby St Luke’s Church.
“The mosque will be open most days,” Mr Islam said. “It is not just for Muslims, it is for the wider community as well.”