The simple and unsurprising answer is YES! However, that does not mean that you have to swim as fast as you can all the time.
As you all know, different types of training sets should be done at different speeds. As the speed of the set changes so too should the ratio between mental and physical effort. So there is kind of a scale from 100% physical effort with very little mental effort to 100% concentration with very little physical effort.
Generally, if you're swimming slow you're doing drills - this means you're trying to change and improve your technique and therefore need to think about what you're doing A LOT! You still have to put in a little physical effort to keep good posture and form, but for the most part it's about getting your brain to tell your muscles what to do and also get feed back from them about how these new techniques feel.
In the middle of the scale would be something like a long 200 pace set. This kind of work requires you to hold on to really good stroke and turn technique while being close to maximum physical effort. If you get this kind of set right then you know you've worked hard!!
Finally there is a real sprinters set. If you're goal is to increase turnover, or kick speed then there are certain sets (or parts of sets) which really require very little thought at all - just ask yourself can my arms/legs go any faster?!!?
These are important things to think about and get right in training. If you're not sure where on the scale the set that you're doing should fall, then ask your coach.