Performance measurement departments are being challenged on several fronts as pointed out by Aite Group analyst Philip Lawton.
(Editor’s Note: When it comes to performance measurement, banks and trading firms face systems integration, data integrity, spreadsheet crawl and client reporting challenges, according to Philip Lawton, a senior analyst for market research firm Aite Group’s Institutional Securities and Investments team.Lawton specializes in regulatory trends, risk management, performance measurement, including Global Investment Performance Standards (GIPS), investment operations, and professional ethics. Before joining Aite Group, Lawton was a partner at Stone House Consulting and the founding head of the Certificate in Investment Performance Measurement (CIPM) program at the CFA Institute. He has served as vice president at State Street Investment Analytics and at Citibank, where he managed performance measurement services for the bank’s US master trust and custody clients. He also spent eight years as an officer in the securities department of The Travelers, where he managed a $9 billion fixed-income portfolio.)
Why are firms having such major problems with the IT infrastructures that support their performance measurement systems?
Well, I think one of the reasons is that many organizations have multiple performance systems in use in either the dedicated asset classes or because they have several work groups conducting performance measurement. They have legacy systems or they’ve acquired specialized systems, so that makes it difficult to integrate material. It certainly makes it difficult to have highly efficient work flows. That integration and workflow streamlining appear to be major concerns in the industry at this point.
What IT management strategies are effective in improving performance measurement systems?
Performance systems are fed by accounting data on the one hand and by reference data on the other. So it’s a matter of getting the reference data, which may be coming from multiple sources, in a format that’s readily interpreted by the performance measurement system and easily integrated into performance reporting against benchmarks, for example, and performance attribution.So, at the very least, it’s a matter of bringing together those two input data streams: the accounting data, which includes positions and cash flows, and also the reference data including benchmarks.
How much of a challenge is it to have multiple systems? From a systems management point of view?
From a systems management point of view, it is challenging. But it may be that the challenges are somewhat more manageable if the systems are used in discrete areas. But then, of course, it’s a question of being able to provide comprehensive performance reports to senior management, who are concerned about overall results for the entire firm.Another related problem is that a great deal of work is still done on spreadsheets. This may be particularly the case when it gets to client reporting where there is a requirement for customization that may not be easily accommodated by performance measurement systems.Spreadsheets really fall outside traditional IT control. In fact, many IT departments disavow any support obligations toward spreadsheets. That’s another area where there has to be an integration. There has to be data flowing into the spreadsheet application so that it can be manipulated. And controlling spreadsheets is a big problem that really falls to the manager of the performance department in the absence of the IT support.
But, unfortunately, a lot of vendors will support spreadsheets. So in some respects, the vendors aren’t helping.
Well, they are trying to help. But that’s sort of throwing in the towel on spreadsheets.
Why is centralization so popular?
It’s popular because with centralization it’s possible to have uniform performance measurement policies and procedures that ensure that the return calculated in one product line or in one part of the world is comparable to a return calculated elsewhere. That would extend to the other demands such as benchmark management policies as well. So centralization is desirable from that point of view.It is also, of course, to the extent that performance management is a specialized discipline that allows the truly knowledgeable, capable, managers to organize and recruit competent analysts, and to redeploy those analysts and those resources as needed. So centralization of a specialized function like this does make a great deal of sense.
The recent Aite Group report described some issues with data integrity. How widespread is the data integrity problem?
I think it’s definitely a concern. It’s usually the accounting data that’s less suspect, but, in that case, if an analyst discovers a dubious return, then the analyst can chase it back upstream to the data inputs and get that corrected.On the reference data side, there are issues there as well. There are occasionally benchmark errors, and there are occasionally benchmark revisions that have to be captured and integrated. Some providers and some benchmark services do data scrubbing as well and it helps.Any time you’ve got a great deal of data coming in from multiple sources with multiple input points, there are going be data integrity problems. It’s actually one of the more important functions served by performance analysts now, which is to keep an eye on it.Returns are calculated by the systems, but the analysts are far more than machine tenders. They watch those returns and identify outliers and identify doubtful results so that they can pursue them and insure that the performance they are reporting is accurate.
What are the major causes for the operational efficiencies that plague client reporting?
Client reporting is a very, very difficult area, and the demands depend to a great extent on the nature of the clientele and the number of clients.For example, in a retail operation there may be thousands or tens of thousands of clients, and, on average, their understanding of investment content may be less sophisticated than one might hope. That creates a set of demands for the retail reporting that is very clear.
“I think that the likelihood of higher spending in performance measurement for systems and for talent is higher if performance professionals demonstrate very clearly how they can add value in the investment decision process.”— Philip Lawton, Aite Group analyst
On the other extreme, there will be institutional, separate accounts where the client is highly sophisticated and really expects not just returns and comparative benchmarks, but also contribution analyses and ex-post analyses. It’s difficult to arrive at a format and at content that is consistently satisfactory to a range of clients, whose expectations are much higher.
Would you say that client reporting is the number one problem for firms in performance measurement?
No, I wouldn’t say that that’s the number one problem. I would say that probably data management, efficiency and consistent quality of return calculations constitute the biggest problem.But client reporting is certainly right up there, and it’s my sense that many very robust performance measurement systems don’t have the flexibility in client reporting that managers would enjoy and would take advantage of, if they could.
What do you think of that push for client reporting standards?
I’m in favor of having at least ethical principles to which one could refer in client reporting.The difficulty there, of course, is that every firm has its own way of doing things; there are the differences in clientele that I mentioned earlier. So, standards that went too far toward specifying exactly what must be included and what sort of explanations must be offered, I have reservations about that.But what I think would be desirable would be to have some general principles to which firms can look in designing reports and developing a client reporting philosophy. It appears to be a very controversial question in the industry as to whether client reporting standards are desirable and worthwhile.I would favor them. But I would not want them to become as extensive and prescriptive as, for example, the GIPS standards are.
If a firm does identify client reporting as a top problem, how extensive are the operational changes that a firm will have to make to revamp their client reporting?
It may be possible to work with the vendors of those performance measurement systems to streamline reporting so that it meets the firm’s and the clients’ expectations.Otherwise, I think it’s a matter of getting data from the performance measurement system to a reporting system. And there are vendors whose expertise and whose primary products are in the area of reporting; in other words they’re not content bound. They’ll report what you have to say and then it’s simply a matter of getting the data from the performance measurement system to the client reporting system that has the flexibility and the professional formatting and the branding capabilities that are desired.
What are the benefits of more interactions between the risk analysts and the performance measurement staff?
I think that the benefits are several. One of them is that performance results—the investment results—provide a very healthy check on the risk manager’s models. They can take the results back and test actual outcomes versus expected outcomes. In fact, I think that is the primary interaction that takes place now. It’s performance measurement providing ex-post data to the risk management area.I think that there’s probably more that can be done. Organizationally, a closer relationship between risk management and performance measurement staffs could be salutary because that it might open career paths not available at present.Performance measurement and risk management professionals have overlapping skill sets—certainly the quantitative, the aptitude, and the ability to think clearly and to understand investment processes and exposures are common to both areas.I don’t think that risk management is mathematically more demanding than performance measurement, though it’s not every performance analyst that can make a transition to risk management. I think that it could enrich the entire organization if such relationships were strengthened.
Is there kind of a turf battle between the performance measurement and risk management people? Is that what’s happening with most firms?
I don’t think so at all. There appears to be much greater and really growing interest in risk on the part of performance analysts, but that’s ex-post risk. So, it’s looking at obvious measures like Sharpe ratios, down-side risk and so forth. I don’t think there’s a turf battle really because the risk management group has the entire domain of ex-ante risk management all to itself.I think where there’s a turf battle perhaps is between risk management and portfolio management, and between portfolio management and performance measurement.In both cases, the analytic work done by risk managers or by performance analysts could be seen as an incursion into the investment decision process that really belongs to portfolio management.
Can you foresee more spending for performance measurement systems and personnel?
I think that the likelihood of higher spending in performance measurement for systems and for talent is higher if performance professionals demonstrate very clearly how they can add value in the investment decision process.They will also have to demonstrate how they can help a firm identify what it does well and what it does not do so well, and, therefore, progressively improve the whole process of portfolio construction and revision.
To learn more from Phil Lawton and hear from other industry professionals, check out our 5th Annual Performance Measurement Conference on March 21st in NYC.
Need a Reprint?
Leave a Reply