It came into existence in 1791 as St. Patrick's Church which was two years before the establishment of the United States hierarchy and twenty-nine years before the institution of the Richmond Diocese. Its first parishioners were French Catholics, compelled to abandon their native land by the French Revolution. In a matter of years, it received some of the earliest Irish Catholic immigrants to the United States.

The original church was built in 1842, but was destroyed by fire in 1856 rendering the building disfunctional for use as a church. In 1858, the present church building was erected. It was dedicated to Mary of the Immaculate Conception, and was the first church to bear the name after the dogma of the Immaculate Conception by Pope Pius IX.

African American Catholics began attending St. Mary's in 1886 where a portion of the choir loft was reserved for them. Subsequently, in 1889 the Josephites began coming from Richmond and by September of that year, St. Joseph's Black Catholic parish was founded with the Josephites serving as priests. Their mission was to serve the spiritual needs of the Black community. Seventy-two years later, 1961, St. Joseph's was merged with St. Mary's. On November 1, 1989, the newly renovated/restored edifice was redidicated with the Most Reverend Pio Laghi, D.D., Apostolic Pro-Nuncio serving as the Principal Celebrant of the Rededication Mass.

Today, St. Mary's Catholic Church is ninety-nine percent African American. The Parish also operates a soup kitchen and provides other outreach to Norfolk's poor and homeless.

On December 8, 1991, the Church of St. Mary of the Immaculate Conception became a Minor Basilica. The date also marked the 200th Anniversary of the church. The offical proclamation was read by Apostolic Pro-Nuncio, Archbishop Agostino Ciacciavillan who also served as the principal celebrant of the Liturgy. Being named a minor basilica is an honor given by the Pope. It becomes a place of pilgrimage and is an honorary title recognizing the distinguished nature of St. Mary's. There are 33 other minor basilicas in the United States, St Mary's is the only one in the Commonwealth of Virginia. December 8, the feast of the Immaculate Conception, is the main celebration at the Basilica.