A compliment was an expression of praise or admiration.
In 2266, the cryptography section of the USS Enterprise was decoding a Romulan transmission from a Bird-of-Prey when Lieutenant Stiles "suggested" giving it to the Vulcan Spock to possibly translate, due to the resemblance between Vulcans and Romulans. Captain James T. Kirk assumed that Stiles complimented Spock on his ability to decode, but Stiles wasn't sure. Kirk then stated to leave his bigotry in his quarters. (TOS: "Balance of Terror")
In 2374, after Kyrians had taken Seven of Nine hostage and entered the USS Voyager's mess hall, The Doctor suggested to Captain Kathryn Janeway that he should go in first to draw any phaser fire if need be. As a result, Ambassador Daleth stated Janeway's crew was heroic. The Doctor clarified that he was invulnerable to phaser fire, but he appreciated the compliment. (VOY: "Living Witness")
As a species, the Clickets evolved a flight-or-fight response to gratitude to the point that compliments were physically painful for them. (LD: "Veritas", "Upper Decks")
In 2380, being thanked for a map of the Romulan Neutral Zone by Captain Carol Freeman led to a brief exchange of phaser fire between the USS Cerritos and the Clicket ship Tweerk. (LD: "Veritas")
In 2382, Freeman repeatedly complimented Clicket captain Zurkel while Lieutenant Commander Steve Stevens broadcast it ship-wide, ending a Clicket invasion of the Cerritos and forcing them to retreat. (LD: "Upper Decks")