Philly is asking residents to rate its services. How do cities measure how they’re doing?

Resident surveys. Analysis of 311 calls. Algorithms that show what residents are saying on social media when they think their city isn’t listening.

Local governments are responsible for spending millions, sometimes billions, of taxpayer dollars. Philadelphia City Council passed a $5 billion budget in June. But deciding where to devote taxpayer money and determining whether services are effective can be difficult.

That’s why Mayor Jim Kenney’s administration this month launched its second “Philadelphia Resident Survey” to ask Philadelphians what its priorities should be next budget year and in the five-year plan it will unveil in March. In the survey, which closes Nov. 3, residents are asked to rate city services and answer questions about their quality of life, including whether they feel safe and how they commute to and from work.

More cities are spending time trying to figure out how they’re doing, said Christiana McFarland, research director for the National League of Cities.

“The emergence of performance offices across the country, the emergence of behavioral insights teams, is leading to cities wanting a better understanding both of the opinion of their constituency and community, as well as how are they performing as a government," she said.

City officials used to depend on sporadic anecdotes to measure resident sentiment — opinions from neighbors, comments at town hall meetings, questionnaires dropped off in mailboxes. Now, said David Johnstone, a board member for the International City/County Management Association, “we have much more sophisticated tools."

“As in many areas of local government,” he said, “that area is fast evolving.”

Communities’ decades-long efforts to survey residents have gotten more professional.

Philadelphia partners with the Temple University Institute for Survey Research to conduct its survey online, by mail, and by phone. The city plans to pay Temple no more than $34,000 through its contract.

Anjali Chainani, director of policy for the mayor’s office, said the city wants the survey to be “a citywide, highly engaged process to ensure we’re understanding what issues are impacting Philadelphians.”

About 7,200 Philadelphians told the Kenney administration through its first resident survey, in 2016-17, that it should put more resources toward fixing streets, improving sanitation, and increasing public safety. Those sentiments were reflected in last year’s budget, which included $178 million to repave streets over six years, funding for new trash trucks, and nearly $100 million over five years for hundreds more police officers.

As local governments increasingly look to data to measure outcomes and inform policy, companies have been cropping up to capitalize on the movement. One, Polco, is a polling company based in Wisconsin that allows cities to pose questions residents can answer online or through an app.

Zencity, a company based in Tel Aviv, Israel, that expanded into the United States in 2018, uses artificial intelligence software to analyze millions of communications to and about a local government. It searches records of calls to 311; emails from residents; public posts, likes, and shares on social media; comments on online news articles; and other communications.

An algorithm classifies the text it “reads” into topics and keeps the types of information a city wants. Cities know what people are saying in real time, but not exactly who is saying it.

“Our goal is to help local governments track resident satisfaction,” said Eyal Feder-Levy, cofounder and chief executive officer.

Measures of resident sentiment have limitations. Wealthier people are more likely to ensure their voices are heard, municipal officials say, so analyzing communications from methods such as social media, emails, and phone calls can exclude poorer residents.

Chicago, San Francisco, and Houston are among the dozens of U.S. cities that have partnered with Zencity, whose efforts have been lauded through a $1 million award from Microsoft. Houston uses Zencity’s software for free through an agreement with Microsoft.

For example, after Zencity’s analysis helped show Houston residents were frustrated by late trash pickups, the city decided to publicize its efforts to hire drivers and buy equipment. Houston spokesperson Alan Bernstein said the platform is among the tools the city plans to continue using — while also noting what it can’t do.

“It doesn’t give us a measure of public opinion,” he said. “It gives us a wonderful measure of the people who are motivated enough to express it on their social media account” or to call the city.

Feder-Levy acknowledged constraints, but noted more people than ever use social media, including some marginalized communities that are more likely to voice opinions there than to contact the city. He sees his platform as an evolution of surveys, which are limited to the relatively few people who fill them out, to the questions cities ask, and to one point in time.

Mark Wheeler, Philadelphia’s chief innovation officer, said he would be willing to try a platform like Zencity’s. But many residents aren’t online and don’t communicate with the city, he said, so Philadelphia relies heavily on in-person conversations with residents and community groups. “I’m concerned we would be missing significant voices,” Wheeler said.

Philadelphia is one city leading in collecting information to measure its performance, said McFarland of the National League of Cities. The city studies the ways people behave to improve how it delivers services.

Philadelphia bakes performance measures into department budgeting, tying money to program outcomes and goals. For example, the city wanted to increase the frequency of restaurant inspections, so it paid for more inspectors to meet its goal, said budget director Marisa Waxman.

Philadelphia officials said they are trying to make departments more efficient and encourage employees to think about challenges more creatively. For instance, the city’s Innovation Fund gives grants to employees and contractors with creative ideas for improving services. In October, the fund awarded more than $45,000, including $7,500 to design a bike corral that will prevent cars from parking in front of fire hydrants while not blocking access.

In the end, the thinking goes, improved performance by city employees should translate into better services for residents.