sources mekorot - Is the name אזולי/אזולאי based on the verse of אשה זנה וחללה לא יקחו

In this answer to Which surnames are Kohanim's and why?, Azulai (however you choose to transliterate it) is a Kohanic surname, based on the the fact that the Rashei Teivos seem to be those of the words in Vayikra 21:7, אשה זנה וחללה לא יקחו, which is talking about a Kohen. However, based on what I read here in Shem Hagedolim (story related in the Rashi script on the right side of the page), there seems to have been some controversy about this.

Does anyone have sources that discuss the original nature of this name, or that discuss this controversy?

Edit: To avoid any confusion, I do not intend to imply anything negative about any people who use this name who are not Kohanim. I am simply asking about the name's history.

Answer

In the preface to his translation of the Chida's diary, Benjamin Cymerman writes as follows:

Ha'im Yosef David Azulai was born in Jerusalem in 5484 (1724), or thereabouts. The exact date is not certain and some historians place his birth as late as 1727. He is descended from a long rabbinic line bearing the name Azulai. The origin of this name is not certain; it has been pointed out that its Hebrew letters are formed from the initial letters of the first five words of verse 7 in Leviticus chap. 21 but this is certainly accidental (and certainly irrelevant). The family had fled from Spain seven generations earlier, and there may be some unknown connexion with the Spanish azul meaning 'blue'.

In the Jewish Encyclopedia article "AZULAI, AZULAY", Louis Ginzberg, Gotthard Deutsch, and A. Porter discuss this as well:

Ḥayyim Joseph David Azulai (see No. 4) derives the family name from the initials of the Hebrew words אשה זנה וחללה לא יקחו ("They shall not take a woman that is a harlot, or profane," Lev. xxi. 7). This derivation, however, is not at all probable; and it is to be presumed that the name refers to a locality in Morocco or in Spain.

This is further discussed in the Jewish Moroccan Archive:

Azoulay is represented here by Moroccan and Palestinian creators. The name is common among Jews in North Africa. It is associated with a variety of meanings: Izil = good in North African Berber, the good one; Tazoulaït = a tribe in Morocco; Azul = blue in Spanish (Azuelos), the one with blue eyes and most interestingly an abbreviation of ‘isha zona vahalala lo yikhu‘ (they shall not wed a prostitute or engaged woman) which implies that Azoulay were priets in an ancient time.

The name may be an Arab/Spanish transformation of Oziel into AzOullah = God beloved. The name may be related to ‘Az’ = ‘strong’ or ‘strength.’ Aziza may be an Aramean transformation of the same name. The names: A’z, O’z, O’ziel are modern Hebrew version used in contemporary Israel.