There are Muslim places that Rambam explicitly noted that it is forbidden for us to visit there under certain conditions. In Iggeret HaShmad, Rambam wrote strong criticism of a fellow rabbi who wished to completely push away all Jews who under violent compulsion accepted Islam. And this is a translated quote of Rambam against that Rabbi:

And he similarly said that the Muslims have idolatry in Mecca and other places, as if the questioner was asking him: ‘Is it permissible to go and participate in the festival of Mecca or not? And similarly did {that Rabbi} state, that the Maniac killed 24000 from Israel, as if the questioner was asking him, does he have a share in the World-to-Come”?

The implication is that in Rambam’s opinion, just as it is forbidden to kill 24000 of Israel, so too is it forbidden to participate in the festival, in the place where the Muslims have idolatry there. And upon those who seek to be lenient is the responsibility to bring clear proof that prayer in a mosque is not in the category of participation in their festival.
And even if we will admit that there is a type of visiting at a place like this that does not fall into the category of participating in their festival, however, I have not heard nor read that the rabbis who gave an overall lenient ruling about being in mosques, instructed those that obey their rulings, not to “participate in festivals” with the Muslims. And regarding he that has the power to protest, but does not protest, see what is stated about him in the Talmud, tractate Shabbat 54b.

There is a religious “Chareidi” Jew who calls himself, Ben Tzion, that documented his visit to the Mosque of “the prophet” in the city of Medina, Saudia Arabia. He provided photographic evidence of himself visiting the mosque while being disguised as an Arab and holding a bag that has his Hebrew name embroidered upon the top edge. The mosque is second in importance in Islam after the Al Charam mosque in Mecca and in this mosque the founder of Islam, that is to say Mohammed is buried.

In modern times, there is a prominent cleric of the Islamic religion that demanded from the Saudi regime to move the grave of Mohammed to an unknown place, because in his opinion, many of his fellow Muslims are stumbling with the sin of idolatry by the means of that grave.

For those who wish to see the article on this: https://www.haaretz.com/middle-east news/1.613692

The reader should be warned that the article about the idolatry issue around the grave comes from a left-wing secular web site and some readers might prefer to avoid the spiritual risks of visiting such a site.

I furthermore have to comment that in my opinion it is a bit weird that even if rabbis would be able to find a solution for the “religious problems” involved in visiting a mosque, that they are not sensitive to the disgrace to Jewish pride by visiting places like these. Namely that these rabbis have no sensitivity to forbid visiting places where they praise with all their might as a prophet, the “small Hitler”, that murdered 24,000 Jews. Behold King David was willing to go to war on a much  smaller slight to national honor (as described in the second book of Shmuel/Samuel chapter10).

Adolph Hitler meeting with Islamic Mufti, Haj Amin Al Husseini

 The Arabs of the land of Israel (Palestine) who were headed by
Mufti Haj Amin Al Husseini collaborated with Hitler to destroy Jews.
According to http://mfa.gov.il/MFA/AboutIsrael/Documents/A%20Century%20of%20Terror.pdf

During WW2, al-Husseini, who had close ties with the Nazi leadership in Germany, had a daily one-hour program in Arabic on Radio Berlin in which he broadcast Nazi propaganda against the Jews and the Allied forces.
Al-Husseini in one of his Arabic broadcasts:
“Arabs, rise as one man and fight for your sacred rights. Kill the Jews wherever you find them. This pleases God, history, and religion. This saves your honor. God is with you” (1 March 1944).
Al-Husseini’s vitriol contributed to anti-Jewish feelings that led to pogroms in Iraq (Farhud) and other Arab countries. He institutionalized Nazi-type hate speech against Jews and, later on, against Israel, that echoes in Palestinian incitement to this day.

According to Their Theology Resurrection of the Dead Is Dependent on Murdering the Jews

The following hadith which forms a part of these Sahih Muslim hadiths has been quoted many times, and it became a part of the charter of Hamas

The Day of Judgement will not come about until Muslims fight the Jews, when the Jew will hide behind stones and trees. The stones and trees will say O Muslims, O Abdullah, there is a Jew behind me, come and kill him. Only the Gharkad tree, (the Boxthorn tree) would not do that because it is one of the trees of the Jews. (related by al-Bukhari and Muslim).Sahih Muslim, 41:6985, see also Sahih Muslim, 41:6981, Sahih Muslim, 41:6982, Sahih Muslim, 41:6983, Sahih Muslim, 41:6984, Sahih al-Bukhari, 4:56:791,(Sahih al-Bukhari, 4:52:177 quoted from  https://en.wikipedia.org/wiki/Islam_and_antisemitismו!”‘

Chief Rabbi Avraham Y. Kook Also Believed in the Concept of Jewish National Pride

A somber gathering assembled in Jerusalem’s Yeshurun synagogue. The large synagogue and its plaza were packed as crowds attended a memorial service for the Jews of Hebron who had been killed during the riots six months earlier, on August 24th, 1929.

On that tragic Sabbath day, news of deadly rioting in Hebron reached the Jewish leaders in Jerusalem. Yitzchak Ben-Zvi, then director of the National Committee, hurried to Rabbi Kook’s house. Together they hastened to meet with Harry Luke, the acting British high commissioner, to urge him to take action and protect the Jews of Hebron.

The Chief Rabbi demanded that the British take severe and immediate measures against the rioters.

What can be done? Luke asked.

Rabbi Kook’s response was to the point. Shoot the murderers!

But I have received no such orders.

Then I am commanding you! Rabbi Kook roared. In the name of humanity’s moral conscience, I demand this!

Rabbi Kook held the acting commissioner responsible for British inaction during the subsequent massacre. Not long after this heated exchange, an official reception was held in Jerusalem, and Mr. Luke held out his hand to greet the Chief Rabbi. To the shock of many, Rabbi Kook refused to shake it.

With quiet fury, the rabbi explained, “I do not shake hands defiled with Jewish blood.” [ excerpt from: http://en.hebron.org.il/history/497 ]