A US appeals court ruled Tuesday that three Muslim men did not have standing to sue FBI agents after being placed on the “no-fly list” following their refusal to become informants for the US government.

The 2nd U.S. Court of Appeals in Manhattan ruled that all 16 officers had qualified immunity.

Circuit Judge Gerard Lynch wrote that the officers could not have violated the rights of Muhammad Tanvir, Jameel Algibhah and Naveed Shinwari because they did not raise objections during their interactions with the officers.

The men were eventually removed from the no-fly list, according to US News & World Report.