DRAWSKO POMORSKIE, Poland—Tanks and armored cars rumbled over the undulating landscape, while smoke from rocket and artillery fire rose into the gray clouds above. Against the deep, dull booms of artillery, machine-gun fire blasted out from right and left. Here in Poland’s northern plains, one of Europe’s historic battlegrounds, the North Atlantic Treaty Organization is carrying out its biggest military exercise in seven years. Some 3,000 troops for 20 nations are taking part in the live-firing operation. The exercise, taking place as NATO looks beyond its combat role in Afghanistan at the end of next year, illustrates a dilemma for the 28-nation alliance. Gen. Philip Breedlove, the U.S. Air Force general who is the alliance’s top military commander, says NATO “will have had 12 years of doing a counterinsurgency fight”—and needs to hone roles it has neglected. NATO allies are looking toward new threats from cyberattacks and challenges to its southern flank posed by an unstable Middle East and North Africa. But at a time of fast-shrinking defense budgets, the exercise shows it is still under pressure to sustain a credible capability to defend the territory of any of its members who asks for help. For NATO’s newer members from the former Soviet bloc, this commitment to mutual self-defense enshrined in Article 5 of NATO’s founding treaty remains central. Their reason: a continuing fear of Russia. It is a position few NATO leaders are willing to proclaim publicly. “This is not about Russia. This is about us. This is about NATO, about our ability…to do this collective defense job,” said Gen. Breedlove in an interview with The Wall Street Journal and Reuters. Volker Rühe, a former German defense minister, said the exercise provided comfort for Poland and the Baltic states by demonstrating “that Article 5 has some meaning, that the core business of NATO hasn’t changed.” Observers from Russia and other countries, including China, attended demonstrations as part as newfound transparency among former adversaries. But defending the territory of a member state against invasion is a step back to the past. “Fear of Russia is not going to be the glue of NATO in the 21st century,” said Mr. Rühe. Anders Fogh Rasmussen, NATO’s secretary-general, said on Wednesday NATO’s role for the future “is to stay prepared for all eventualities.” One real potential test is to the south. Ian Kearns, director of a security-focused think tank of former senior European officials, called the European Leadership Network, said the increasingly unstable neighborhood on NATO’s southern flank—in the Middle East and North Africa—will likely occupy more alliance time. NATO’s next big exercise, in 2015, which senior officers say may involve tens of thousands or more troops, will be focused on the Mediterranean. Gen. Breedlove said he hoped that one would “focus on more high-end, kinetic, collective-defense exercises.” Another big challenge as defense budgets shrink is to get NATO to cooperate better—not only on exercises and operations but also in buying defense equipment. The U.S. is sending signals it wants Europe to carry more of its own burden. To the live-fire exercise, the U.S. sent just 160 troops—though others were deployed back in the mobile headquarters in Latvia. Some countries are already giving up military capabilities to save money—relying on other nations to watch their backs while they specialize on other tasks. The Netherlands, for example, no longer has tanks, depending on Germany’s force. Air patrols are carried out over the Baltic states—Latvia, Lithuania and Estonia—by other NATO air forces because the three have focused on their land forces. For many countries, this trend will be tough to accept, Gen Breedlove said. “Many NATO nations are already realizing that they can’t have a full-spectrum military,” he said. Many specialists say what has happened is a start—but for Europe to preserve an effective military, such steps will have to be far more radical. Mr. Rühe suggests the U.K. and France, with Europe”s most capable militaries, should coordinate major programs—such as aircraft carriers. They should even coordinate their highly sensitive nuclear forces. “I think it will happen but not for another 10-15 years,” he said. The future may also see German, Spanish and Italian planes flying off British and French aircraft carriers. The trouble now is that nations don’t see eye-to-eye on foreign and security policy and a reluctant ally could block others from taking action. Germany’s reluctance to use force means Europe’s biggest economy isn’t viewed as a reliable partner by some allies, Mr. Rühe said, particularly after its refusal to take part in the alliance’s 2011 military operation over Libya. If NATO’s European allies aren’t willing to spend more, they will have to share more or lose effectiveness, he said. “In those circumstances, you are forced toward a common foreign policy,” he said. In today’s Europe, that seems a long way off.