For trumpets, it's a matter of having piston valves, which have a short actuation length and can be moved very quickly. In addition, having a shorter air column allows the instrument to speak faster and more easily, aiding in rapid passages.
However, that's not to say that trombones (with slides) and horns (with rotary valves) can't move quickly. In the late Renaissance when composers started writing suggestions of instruments to play certain parts, trombones often featured along with violins and cornetti, both of which were extremely agile instruments. In the Baroque and early Classical, (natural, i.e. valveless) horns were called upon to play quite remarkably intricate lines.
Some examples-
https://www.youtube.com/watch?v=Tq7-4SSxP50 (note that although the music is period, the instruments are using modern chromium-plated slides and the players are of a very high level)
https://www.youtube.com/watch?v=wkIvJCEX5Nc (without valves, the players use their embrouchre to choose which partial of the natural horn to use and their hard variably inside the bell to tune. Trills are done with the lips only, no valves)
https://youtu.be/b0rb1IXZI9c?t=126 (Arthur Pryor is generally considered to have been one of the most virtuosic trombone players of all time)
https://youtu.be/MCOMdGgreFM?t=253 (Max Acree is a pretty badass local jazz trombonist)
Edit: Note that, and this goes into the follow up question, these complicated things on the natural horns are only possible because they are intentionally written to be accessible by avoiding notes which would be difficult or impossible to reproduce on the natural horn. Similarly for a trombone, writing something that would require rapidly moving the slide back and forth from full extension to all the way in would probably not go well. For both cases, moving into higher partials where the partials are closer together reduces this difficulty, at the cost of physical strain. For a valved instrument like trumpet or modern horn, there is not much difference between pressing any one of the valves, although very distant keys may require lots of changes in fingerings and tax less competent players mentally.
Regarding agility in keys, this is partially a matter of physical construction and partially of pedagogy.
For one part, all necked strings are played from the concept of 'putting finger down raises pitch', ergo, fundamentally thinking from the perspective of 'raising' or 'sharping', rather than flatting. When a cellist wishes to play the first D on the instrument, they press the C string down onto the fingerboard a certain distance down the length of the fingerboard, thus shortening the string and raising its pitch.
All modern brass and woodwinds (not some historical ones, mind you) function under the opposite assumption- 'putting finger down lowers pitch', thus fundamentally think from the perspective of 'flattening' or 'lowering' pitch. When a trumpet player depresses a valve, the air is sent through an extra bit of tubing corresponding to the desired change (e.g. lower by wholestep, halfstep, or step and a half, combining valves to lower further).
In effect, most instruments either work by accessing a certain overtone/partial or string and altering the length of the instrument or string to change the pitch relative to that partial. When a trombone plays a Bb below middle C or a G below middle C, he/she is still accessing the 4th harmonic of the instrument, just the sounding length is changing via the slide. For a string player, strings have a fixed length and then are made shorter to raise pitch, usually by placing a finger on the string to shorten it (some instruments stretch the string, such as Asian zithers like dan tranh and gayageum, but this is less common).
As a result of both the instruction of players and the construction of instruments, it has evolved that strings are generally written for in sharp keys while brass and woodwinds are written for in flats.
Note that there are some edge cases- the ophicleide (metal precursor to tuba featuring keys) uses an ascending system, where putting the finger down raises the pitch on all but one of the keys. There were also ascending-system horns made in the 19th century but the 'descending' approach won out.
Keyboard instruments are 'agnostic' in this approach, as sharps and flats are conceptually identical. Several DAWs even label their piano rolls with sharps only, leading to quite a few EDM tracks in the key of "A# Major".
Concert (pedal) harps are an interesting edge case in that while they feature strings, which can only be raised in pitch, they normally operate with the pitch raised on all strings from their sounding length by a halfstep (in effect, with tuning mechanisms completely engaged, a harp would be in diatonic Cb or B Major {it sounds strange, but it's actually kinda easier to think of it as Cb, so just roll with it...}). Thus the harpist conceptually thinks of the instrument as a configurable octotonic instrument, where notes C through B can each be individually flatted or raised by a halfstep each (thus each note name may be flat, natural, or sharp across the range of the instrument)... thus making it not just key agnostic, but to some extent scale agnostic, meaning once configured, there is no difference in how they interact with the strings between C Major, minor, harmonic minor, wholetone, any of the church modes, and even non-standard scales... so long as they can be achieved using all 7 notes in one tuning or another (so a pentatonic glissando, for example, is not conventionally possible).
Lever harps feature only half-step mechanisms may be tuned with B-flats and E-flats, then raise those notes using the sharpening lever, thus allowing notes to be flatted. Sharpening levers only apply to the individual string, so a lever harp could theoretically have different notes tuned to different things in different octaves, such as a scale that doesn't repeat at the octave but, for example, every other octave. Whether or not lever harpists conceive of their relationship with sharps and flats different than pedal/concert harpists, I don't know, but I would suggest that they may be more at home in common neutral keys (Bb, F, C, G, D) due to the relative simplicity of their instruments' construction.