In recent appearances, President Obama has suggested that it’s time for Washington to confront the intense polarization and incivility that mark our politics these days.

His first sally was his back-and-forth with the House Republican caucus at its retreat in Baltimore. He followed that a few days later with a speech to the National Prayer Breakfast decrying the “erosion of civility” in Washington and the inability of politicians in an increasingly partisan culture to listen to one other. “Those of us in Washington are not serving the people as well as we should,” he said.

Lots of ordinary Americans would agree with those lofty sentiments. But what’s notable is the growing concern even in Washington that, when it comes to the actual business of governing, the nation’s political leaders appear so riven with conflict that they’re unable to move forward on anything. Both Democrats and Republicans welcomed the President’s visit with the House Republicans as a first, tentative step in trying to reduce partisanship.

Moves like these are important gestures. But intense partisanship is deeply rooted in the body politic now. Even if the entire leadership at both ends of Pennsylvania Avenue were suddenly to embrace one another in honest fellowship, there would still be a long way to go in reducing polarization. That is because much of our political culture now works to drive people apart, not bring them together.

To begin with, we face a somewhat confusing paradox: In terms of electoral politics, the country is closely divided between left and right, with one side or the other gaining a majority depending on where independents choose to alight on election day. Yet in terms of political values, the nation is above all pragmatic and moderate, caring less about ideology than about what works.

The problem is that too much in politics — the extent to which congressional districts lock in a single party’s dominance, the increasing importance of primaries dominated by the ideologically driven voters in both parties, and hence the growing ideological homogeneity of both parties’ leadership — works to favor division, not pragmatism.

The result is that politics now drives policy on Capitol Hill. Every vote is looked upon as a political vote, with members of Congress asking themselves not, “What’s best for the country?” but, “How do we put the other guys on the spot and advance our own partisan interests?”

This trend toward the extremes has also been driven by political developments in the country at large. Demographic trends — the migration of African-Americans out of the South, the tendency of people of similar class and ethnic background to cluster together — have created communities and even regions that are dominated by one party or the other. This has been echoed by an explosion of advocacy organizations, so that groups that used to create consensus out of wildly disparate views no longer do so.

The political parties, which once forged consensus platforms at conventions that were notable for their diversity, now cater to their ideological activists. Advocacy associations — whether focused on the environment, agriculture, health, or whatever — that once had to build an agenda acceptable to a diverse membership, now are so narrowly aimed that they feel free to pursue their parochial points of view.

The media, too, have fragmented. Americans get their information from a bewildering array of sources, and these days need never be troubled by reporting or analysis that doesn’t agree with their own preconceived views of the world. Punditry and commentary are what rule the media-sphere now, not hard reporting, and much of it is ideologically driven. There are very few prominent media voices pushing political Washington toward the center.

All of this has made it hard for fair, open-minded and centrist politicians to gain any footing, and has pushed their counterparts in the population at large to withdraw from a politics they see as increasingly nasty, closed-minded and unattractive.

If there’s a solution, it lies with ordinary Americans willing to stand up and say “Enough’s enough!” The President and other political leaders can certainly try to change the tone in Washington, but they have an uphill battle to fight unless enough Americans make it clear that they are so tired of polarization, they’ll set their own ideological prejudices aside and place a premium on politicians who demonstrate they know how to work with people who don’t agree with them.

Lee Hamilton is director of the Center on Congress at Indiana University. He was a member of the U.S. House of Representatives for 34 years.