Numbers don’t speak for themselves: Data and Transparency


A Draft Opinion Piece – Introductory Context

I have been working on and/or researching the application of digital technology to film industry business models since 2008. My current research on the conceptualisation, materialisation and management of risk in film and integrated creative content businesses, necessarily deals with many aspects of digital tools’ role in the industry. In considering one aspect of this project, namely the lack of established and uniform understanding of risk amongst various market actors outside a relatively narrow, highly engaged core, I was struck by several related features of current industry relevant literature. One is how little industry commentary has developed its thinking on film industry business models and the availability of digital data. A second is how poorly the wealth of academic research on the related subjects has been disseminated and/or taken up. A third is the lack of interaction amongst, or at least common appreciation of, the diverse disciplinary outputs attending to different aspects of the issue at hand.

There still appears to be a number of common problems or ambiguities in trade press and industry reports regarding how the potential use of film data is presented. These issues could be greatly diffused were existing research to be more widely absorbed. This lack is a little surprising given that public and industry engagement and impact are measures of academic success. The potential that such misunderstandings around film business analysis delay market development or put off potential outside interest, means that this is a serious issue we need to address.

The problems I am thinking of, and which are deeply interconnected are:

  • A simplified comparison between the access to, and use of, large datasets by major studios and international companies, with the potential benefits to the independent industry.
  • The conflation between consumer demand as expressed by anticipated BO returns, and sales of international rights / distribution MGs and their impact on ROI for participants.
  • A contradictory standpoint which foregrounds a radical disruption to independent industry organisation and revenue flows e.g. the collapsing of windows and territories and thereby the process of forecasting ultimate revenues; and a substantial reliance on limited datasets of budgets, BO,estimated marketing spend, and dated heuristics as a guide to analysing profitability.
  • A lack of acknowledgment or attention to financial structure as the determinant of successful film investment, as opposed to evaluation through reduction to proxy measures of winning / losing with respect to BO data.
  • A general absence of recognition for research already well established in cultural economics and marketing dealing with film revenue return distributions.

A tradition of difficulty

The film industry has to combat significant barriers to acceptance of its quantitative expression. Stereotypes of opaque accounting, poor reporting, and distributor’s reliance on late payment of debts are not aided by the development of further numbers-led agendas with a  lack of clarity, or misrepresented simplicity. So we must pursue solutions to a broader, more detailed understanding of the film industry’s complexities.

Although an impact agenda does exist for researchers, there are also counterproductive disincentives for experts in academia or in the professions to ensure the widest, deepest practitioner audience to adopt their knowledge. Understandably the relatively small coterie of film financiers, lawyers and accountants are typically, exceptionally busy. They are gainfully employed in executing industry arrangements, therefore unable to repeatedly deliver specialist advice in public forums. Academics have more duty and motivation to do so, but the structure of performance evaluation and career development militates against inter or cross-disciplinary fertilisation. The work of econometricians and cultural economists, creative industries scholars, researchers of media industries and cinema studies or cultural industries as relevant to the film industry, is rarely pulled together.

There are rare and valuable collections that provide meeting points around particular subjects, often driven by conferences e.g. Dempster’s book on risk and the art world and Hjort’s edited volume on film and risk containing Grantham’s expert chapter. Often though, given the limited scope of introduction and epilogues and their irregular publication, these collaborative endeavours are unable to coherently synthesise the contribution of multiple disciplines to a single area. Although Open Access has improved dissemination of findings, the time taken for publication of peer reviewed research in fast changing fields like the film industry mean that research is often lost to practice.

How then to ensure market actors access cutting-edge research in a timely fashion to gain an informed, holistic understanding of the business? How to avoid the popular dominance of introductory and more simplistic works, cruelly but often truthfully described as “written for novices by outsiders”? Academics gain substantially from industry access. Whilst insights from research projects are usually felt by the specific industry partners, especially in applied fields like digital marketing, the wider industry rarely reaps the benefit of knowledge generation. Perhaps, as a small starting point, one way of dealing with these challenges is to actively engage film industry institutions that are dealing with open research questions, and point out areas where existing research can help. Consider the continuing demand for independent producers’ access to VOD data and the crucial, but typically absent qualification of how such data could and should be used.

Practitioners / Researchers: Drowning in a flood of data / speaking in a vacuum

Increased availability of information, from audience engagement data to HE revenues, is a good thing, most certainly from a researcher’s perspective. Being better informed about any market in which you are operating is surely a benefit. Yet broad brushstroke assessments of access to data, broadly understood, as panacea for lack of control or career unsustainability in independent filmmaking, require a great deal of development and definition to be productive. Knowledge transfer about market demand e.g. consumption information from the European Audio-Visual Observatory, or standard deal terms and necessary workload provided by training schemes like Inside Pictures; Build Your Audience and Make Your Market, enhances the potential for success in an industry where deception and opacity have become stereotypes.

However, continued lack of detail in independent industry calls for access, for example to VOD data, means there is a threat that the underlying dynamics of the industry, which may be viewed through the conceptual lens of risk, are omitted or misrepresented, adding to a detrimental lack of wider understanding of the business.

This is a subject with many parts, which are often condensed to suggest that simple access to numbers will enable improvement in the financial performance of independent producers. Typically, such positions make direct comparisons between the use of large datasets by major studios and other global market actors, and the benefits potentially accessible to independent producers. E.g. Data mining the relationship between different film variables (cast, genre, down to specific script points, or online anticipation) and past BO performance, to inform development, investment, production and distribution decisions.

There are a number of issues that need to be strategically addressed to unpack how different market actors can and do utilise different kinds of data, in order that viable propositions for change can be put forward. Methodology is an extremely important and complex feature of this area, which is regularly overlooked in thrall to the power of large numbers and an acceptance that all quantitative analysis is de facto valid. A light needs to be shone on the internal workings of how data-based decisions are made, the external context of those decisions, and, indeed how those two motivations interact. Sometimes the implications of latter can undermine the validity of the former. The classic Hollywood example being reliance on sophisticated portfolio modelling and then investing via a single picture greenlighting system.

So, some important issues to consider:

  • Studio business models and thus their use of audience and revenue data are substantively different from the concerns of independent filmmakers.
    • The most advanced data-intensive modelling currently adopted may be argued to be the use of artificial neural networks in examining the probability distribution of potential revenue outcomes from different scripts / packages. Its adoption in development is primarily defensive i.e. to avoid significant losses (see Epagogix etc.) rather than to build projects.
    • The most effective and accurate use of revenue modelling may be argued to be the application of digital word of mouth analysis, which can be utilised to refine and target marketing and distribution campaigns, see Asur and Huberman 2010, and Wiertz 2015.
    • Whilst large MNCs are able to run projections for ultimates and anticipate demand via purchase or streaming data, these type of considerations are only part of the process by which management decisions are made. Also feeding in to debates are HR concerns relating to managing talent relations; short term growth goals linked to share price; long term strategies, answerable to oversight at ownership level (Comcast, Sony, Viacom). These are non-specific generalisations, but the point is to illustrate that the availability of data does not mean all market activity is taken according to rational economics and probabilistic risk. Information asymmetry certainly exists between large and small companies, it provides a definite advantage, but the details of application matter.
  • In seeking to approximate the information advantage larger competitors enjoy, independents must know what can be achieved and how, to ensure they are being efficient in allocating their resources.
  • Typically the most important sources of income for independent films are payments from sales agents and distributors, not the filtering through of overages. Thus knowledge of the financial performance of other films from consumer sources is useful insofar as it can be analysed in relation to likely returns to distributors and thus their derived demand for rights to a film (parsed through the calculation process of sales estimates). Given the breakdown in windows and territories this cross-calculation process is ever more uncertain. It is also reliant upon the common acceptance of past performances reliably indicating future events, an assumption a vast swathe of data-intensive film economics literature negates, and the organisational structure of the independent industry and film financing is arranged against. More importantly though is that the strategic use of such data is most viable when allied to deal term information.
    • E.g. How does a rights-holder weigh up distributor offers, and self-release options in a way that maximises the use of performance data on certain types and releases of film, without considering the allocation of those returns?
  • This is a difficult area. The hot topic presently is that rapidly growing SVOD services do not disclose viewing data, pay a flat fee, and often buy packages of films from licensors. This makes evaluating a film’s performance challenging. The audience is unknown, the proportion of a fee allocated to a specific film in a bundle is likely to be unknown and often deal term details cannot be divulged even to prospective shareholders. Many independents are forced to wait to see which titles are relicensed for a second round to garner some indication of what films are popular on particular platforms.

TVOD data is clearer, it can be detailed if derived from a white label service like VHX, or at least sales can be discoverable by reverse engineering collection account receivables with standard price points and terms, if distributors’ reporting is limited. Delayed and opaque or confusing reporting can be the norm for independent producers own films, adding the barrier of confidentiality restraints over private business information from other parties complicates the situation further. For example the most well supported endeavour in this field, the Sundance Transparency Project has collected only 60 films worth of, what they deem “detailed” reporting and their demonstration analytics do not break down revenue between broadcast and SVOD.

  • The initial findings of the Sundance Institute project illustrate the pros and cons of this kind of work. The aim to “empower filmmakers” through the ability to “model out future or hypothetical film projects and test hypotheses about distribution strategies” and thereby “build better business models” is admirable. Clearly this ambitious goal is currently an overreach without both a significantly greater volume of much more granular data and a detailed articulation of how such aims might be pursued. It is crucial that quantitative methodologies are fully explained and the differences between descriptive and predictive data uses foregrounded.

For instance the example Sundance use of a filmmaker viewing the median average of 26 films’ revenue and P&A information, in the context of the use of language such as hypothesis testing to drive business models, risks overstating the robustness of the quantitative tool as it stands. This clarity in presentation and thus public / industry perception is important because there is a high cost of time and effort for practitioners to engage in new initiatives. As Sundance note: “it takes at least 3-4 months of outreach to gather data from filmmakers, which is understandable as the work-to-reward ratio is too high”. This point is supported by Nesta R&D findings indicating that whilst producers would like more data and better reporting, their available time to access and analyse such data for action is extremely limited. Therefore to get the fullest support from producers, for them en masse to persuade the sales agents, distributors, platforms and retailers to OK the sharing of such data, there has to be a clear, accurate, and realistic path for its use.

It is therefore right, though it perhaps should be re-foregrounded and prioritised, that Sundance also note their endeavour as a first step, as a guide to market trends, and most concretely, a viable project to engender data standardisation and reporting.

This is not at all to say that access to data is worthless for independents, wrong to pursue, or that digital distribution does not generate a scale of data of a different order to that of previous eras, and with potentially great benefit. Rather, the point is to advocate for considered, specific analysis of application, with recognition of current barriers and limitations to the application of information.

Researchers have a significant responsibility here to ensure current, methodologically sound results are disseminated. Often, managerial implications are added to journal articles as an afterthought, without due consideration for the practical uptake of such recommendations, not least the reading of them in the first place. When there is a wealth of research on what lessons can be learned from past performance data, and to what degree of confidence predictions can be made, it should not go to waste. Given the need for public impact of academic research, especially for those with feet both in professional schools or practice based institutions and industry, there is great scope for interaction.

In the spirit of contributing to this end, a project I have been working on for some time may well fit under the rubric of the Transparency Project’s goal to “help filmmakers be more creative and efficient in funding, marketing and releasing their work.” The affordances of specialised search engines e.g. price comparison websites, Skyscanner et al. could be applied (assuming T&C issues are addressed) to enable the scaled interrogation of databases such as Cinando, Screenbase, Mavise, and Lumiere for the tracking of deals per film. This information on who buys what, where, could then be mapped to the core metrics summarised currently with the Transparency Project, or larger market trends, so that the relative demand in business-to-business markets of particular film types can be analysed and filmmaker strategies adjusted accordingly. The ultimate aim being to approximate the potential revenue for rights holders, benchmark indicative MGs (announced / leaked / or very unlikely, accurately shared) and thereby strategize the most advantageous finance and release plan for a film.

A refocusing of attention onto industry market transactions based on derived demand, removes from the equation concerns over probabilistic inferences of future BO performance (and anticipated trickle down effects to HE) based on past averages. If filmmakers are to benefit from an increase in data availability it is crucial that the most impactful uses of data are prioritised and potential confusion over solutions to radical uncertainty over revenue distributions for unique products is addressed. Some extremely large corporations may well have end-to-end traceable data pathways e.g. those ascribed to Alibaba, with which more revolutionary prediction goals may be met, this is not the sort of power available via full development of the Transparency Project or related initiatives.

Examining and setting out how different types of film industry practitioners may best use data e.g. for creative recommendations of release windowing, comparison of potential public interest per territory with historical sales deals and distributor appetite, brings quantitative information into conversation with company networked relations. This broad perspective on the consideration of quantitative data in relation to managing risk in film business can adapt ideas from human business initiatives that are being developed in relation to creative endeavours. It is something that researchers need to get better at.




Cover photo by Amber Case under CC via Flickr

Top Photo from OMDb under CC

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s