Jorge Casillas and Francisco J. Martínez-López (Eds.) Marketing Intelligent Systems Using Soft Computing: Managerial and Research Applications Studies in Fuzziness and Soft Computing, Volume 258 Editor-in-Chief Prof. Janusz Kacprzyk Systems Research Institute Polish Academy of Sciences ul. Newelska 6 01-447 Warsaw Poland E-mail: kacprzyk@ibspan.waw.pl Further volumes of this series can be found on our homepage: springer.com Vol. 243. Rudolf Seising (Ed.) Views on Fuzzy Sets and Systems from Different Perspectives, 2009 ISBN 978-3-540-93801-9 Vol. 251. George A. Anastassiou Fuzzy Mathematics: Approximation Theory, 2010 ISBN 978-3-642-11219-5 Vol. 244. Xiaodong Liu and Witold Pedrycz Axiomatic Fuzzy Set Theory and Its Applications, 2009 ISBN 978-3-642-00401-8 Vol. 252. Cengiz Kahraman, Mesut Yavuz (Eds.) Production Engineering and Management under Fuzziness, 2010 ISBN 978-3-642-12051-0 Vol. 253. Badredine Arfi Linguistic Fuzzy Logic Methods in Social Sciences, 2010 ISBN 978-3-642-13342-8 Vol. 254. Weldon A. Lodwick, Janusz Kacprzyk (Eds.) Fuzzy Optimization, 2010 ISBN 978-3-642-13934-5 Vol. 255. Zongmin Ma, Li Yan (Eds.) Soft Computing in XML Data Management, 2010 ISBN 978-3-642-14009-9 Vol. 256. Robert Jeansoulin, Odile Papini, Henri Prade, and Steven Schockaert (Eds.) Methods for Handling Imperfect Spatial Information, 2010 ISBN 978-3-642-14754-8 Vol. 257. Salvatore Greco, Ricardo Alberto Marques Pereira, Massimo Squillante, Ronald R. Yager, and Janusz Kacprzyk (Eds.) Preferences and Decisions,2010 ISBN 978-3-642-15975-6 Vol. 245. Xuzhu Wang, Da Ruan, Etienne E. Kerre Mathematics of Fuzziness – Basic Issues, 2009 ISBN 978-3-540-78310-7 Vol. 246. Piedad Brox, Iluminada Castillo, Santiago Sánchez Solano Fuzzy Logic-Based Algorithms for Video De-Interlacing, 2010 ISBN 978-3-642-10694-1 Vol. 247. Michael Glykas Fuzzy Cognitive Maps, 2010 ISBN 978-3-642-03219-6 Vol. 248. Bing-Yuan Cao Optimal Models and Methods with Fuzzy Quantities, 2010 ISBN 978-3-642-10710-8 Vol. 249. Bernadette Bouchon-Meunier, Luis Magdalena, Manuel Ojeda-Aciego, José-Luis Verdegay, Ronald R. Yager (Eds.) Foundations of Reasoning under Uncertainty, 2010 ISBN 978-3-642-10726-9 Vol. 250. Xiaoxia Huang Portfolio Analysis, 2010 ISBN 978-3-642-11213-3 Vol. 258. Jorge Casillas and Francisco J. Martínez-López (Eds.) Marketing Intelligent Systems Using Soft Computing: Managerial and Research Applications, 2010 ISBN 978-3-642-15605-2 Jorge Casillas and Francisco J. Martínez-López (Eds.) Marketing Intelligent Systems Using Soft Computing: Managerial and Research Applications ABC Editors Dr. Jorge Casillas Department of Computer Science and Artificial Intelligence Computer and Telecommunication Engineering School University of Granada Granada E-18071 Spain E-mail: casillas@decsai.ugr.es Dr. Francisco J. Martínez-López Department of Marketing Business Faculty University of Granada Granada, Spain E-18.071 E-mail: fjmlopez@ugr.es ISBN 978-3-642-15605-2 e-ISBN 978-3-642-15606-9 and Department of Economics and Business – Marketing Group Open University of Catalonia Barcelona, Spain E-08.035 E-mail: fmartinezl@uoc.edu DOI 10.1007/978-3-642-15606-9 Studies in Fuzziness and Soft Computing ISSN 1434-9922 Library of Congress Control Number: 2010934965 c 2010 Springer-Verlag Berlin Heidelberg  This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typeset & Cover Design: Scientific Publishing Services Pvt. Ltd., Chennai, India. Printed on acid-free paper 987654321 springer.com Foreword Dr. Jay Liebowitz Orkand Endowed Chair in Management and Technology University of Maryland University College Graduate School of Management & Technology 3501 University Boulevard East Adelphi, Maryland 20783-8030 USA jliebowitz@umuc.edu When I first heard the general topic of this book, Marketing Intelligent Systems or what I’ll refer to as Marketing Intelligence, it sounded quite intriguing. Certainly, the marketing field is laden with numeric and symbolic data, ripe for various types of mining—data, text, multimedia, and web mining. It’s an open laboratory for applying numerous forms of intelligentsia—neural networks, data mining, expert systems, intelligent agents, genetic algorithms, support vector machines, hidden Markov models, fuzzy logic, hybrid intelligent systems, and other techniques. I always felt that the marketing and finance domains are wonderful application areas for intelligent systems, and this book demonstrates the synergy between marketing and intelligent systems, especially soft computing. Interactive advertising is a complementary field to marketing where intelligent systems can play a role. I had the pleasure of working on a summer faculty fellowship with R/GA in New York City—they have been ranked as the top interactive advertising agency worldwide. I quickly learned that interactive advertising also takes advantage of data visualization and intelligent systems technologies to help inform the Chief Marketing Officer of various companies. Having improved ways to present information for strategic decision making through use of these technologies is a great benefit. A number of interactive advertising agencies have groups working on “data intelligence” in order to present different views of sales and other data in order to help their clients make better marketing decisions. Let’s explore the term “marketing intelligence”. The Marketing Intelligence & Planning journal, published by Emerald Publishers, “aims to provide a vehicle that will help marketing managers put research and plans into action.” In its aims and scope, the editors further explain, “Within that broad description lies a wealth of skills encompassing information-gathering, data interpretation, consumer psychology, technological resource knowledge, demographics and the marshalling of human and technical resources to create a powerful strategy.” Data interpretation seems to be at the intersection of “marketing” and “intelligence”. By applying advanced technologies, data can be interpreted and visualized in order to enhance the decision making ability of the marketing executives. Certainly, blogs and social networking sites are rich forums for applying mining techniques to look for hidden VI Foreword patterns and relationships. These patterns may enrich the discovery process and allow different views, perhaps those unexpected, from those initially conceived. In Inderscience’s International Journal of Business Forecasting and Marketing Intelligence, the focus is on applying innovative intelligence methodologies, such as rule-based forecasting, fuzzy logic forecasting, and other intelligent system techniques, to improve forecasting and marketing decisions. In looking at the Winter 2010 Marketing Educator’s American Marketing Association Conference, there are a number of tracks presented where the use of intelligent systems could be helpful: Consumer behavior, global marketing, brand marketing, business-tobusiness marketing, research methods, marketing strategy, sales and customer relationship management, service science, retailing, and marketing & technology. Digital-centered marketing where one takes advantage of such digital marketing elements as mobile, viral, and social marketing channels is a growing field that can apply the synergies of marketing and intelligent systems. Positions for Directors of Marketing Intelligence are also appearing to be the champions of new marketing methods. Gartner Group reports, such as the August 2008 report on “Social Media Delivers Marketing Intelligence”, are further evidence of this evolving field. In a recent report of “hot topics” for college undergraduates to select as majors in the coming years, the fields of service science, sustainability, health informatics, and computational sciences were cited as the key emerging fields. Certainly, marketing intelligence can play a key role in the service science field, as well as perhaps some of the other fields noted. In May 2008, there was even a special issue on “Service Intelligence and Service Science” published in the Springer Service-Oriented Computing and Applications Journal. In July 2009, there was the 3rd International Workshop on Service Intelligence and Computing to look at the synergies between the service intelligence and service sciences fields. In the years ahead, advanced computational technologies will be applied to the service science domain to enhance marketing types of decisions. In 2006, I edited a book titled Strategic Intelligence: Business Intelligence, Competitive Intelligence, and Knowledge Management (Taylor & Francis). I defined strategic intelligence as the aggregation of the other types of intelligentsia to provide value-added information and knowledge toward making organizational strategic decisions. I see strategic intelligence as the intersection of business intelligence, competitive intelligence, and knowledge management, whereby business intelligence and knowledge management have a more internal focus and competitive intelligence has a greater external view. Marketing intelligence seems to contribute to both business and competitive intelligence—helping to identify hidden patterns and relationships of large masses of data and text and also assisting in developing a systematic program for collecting, analyzing, and managing external information relating to an organization’s decision making process. I believe that this book sheds important light on how marketing intelligence, through the use of complementary marketing and intelligent systems techniques, can add to the strategic intelligence of an organization. The chapters present both a marketing and soft computing/intelligent systems perspective, respectively. I commend the editors and authors towards filling the vacuum in providing a key reference text in the marketing intelligence field. Enjoy! Preface The development of ad hoc Knowledge Discovery in Databases (KDD) applications for the resolution of information and decision-taking problems in marketing is more necessary than ever. If we observe the evolution of so-called Marketing Management Support Systems (MkMSS) through time, it is easy to see how the new categories of systems which have appeared over the last two decades have led in that direction. In fact, during the eighties, the inflection point was set that marked a transition stage from what are known as Data-driven Systems to Knowledge-based Systems, i.e. MkMSS based on Artificial Intelligent (AI) methods. The popular Marketing Expert Systems were the first type in this MkMSS category. Then, other new types within this category appeared, such as Casebased Reasoning Marketing Systems, Systems for the Improvement of Creativity in Marketing, Marketing Systems based on Artificial Neural Networks, Fuzzy Rules, etc. Most of these systems have been recent proposals and, in any case, their application is still scarce in marketing practical and, specially, academic domains. Anyhow, we have noticed a clear greater interest and use of these Knowledge-based Systems among marketing professionals than among marketing academics. Indeed, we perceive a notable disconnection of the latter from these systems, who still base most of their analytical methods on techniques belonging to statistics. Doubtless, this fact contributes to these two dimensions of marketing—i.e. the professional and the academic—grow apart. During the years that we have been working on this research stream, we have realized the significant lack of papers, especially in marketing journals, which focus on developing ad hoc AI-based methods and tools to solve marketing problems. Obviously, this also implies a lack of involvement by marketing academics in this promising research stream in marketing. Among the reasons that can be argued to justify the residual use that marketing academics make of AI, we highlight a generalized ignorance of what some branches of the AI discipline (such as knowledge-based systems, machine learning, soft computing, search and optimization algorithms, etc.) can offer. Of course, we encourage marketing academics to show a strong determination to approximate AI to the marketing discipline. When we talk about approximation, we refer to going far beyond a superficial knowledge of what these AI concepts are. On the contrary, we believe that multidisciplinary research projects, formed by hybrid teams of marketing and artificial intelligence people, are more than necessary. In essence, the AI discipline has a notable number of good researchers who are interested in applying their proposals, where business in general, management VIII Preface and, in particular, marketing are target areas for application. However, the quality of such applications necessarily depends on how well described the marketing problem to be solved is, as well as how well developed and applied the AI-based methods are. This means having the support and involvement of people belonging to marketing, the users of such applications. Considering the above, this editorial project has two strategic aims: 1. 2. Contribute and encourage the worldwide take-off of what we have called Marketing Intelligent Systems. These are, in general, AI-based systems applied to aid decision-taking in marketing. Moreover, when we recently proposed this term of Marketing Intelligent Systems, we specifically related it to the development and application of intelligent systems based on Soft Computing and other machine-learning methods for marketing. This is the main scope of interest. Promote the idea of interdisciplinary research projects, with members belonging to AI and marketing, in order to develop better applications thanks to the collaboration of both disciplines. This book volume presented here is a worthy start for these purposes. Next, we briefly comment on its structural parts. Prior to the presentation of the manuscripts selected after a competitive call for chapters, the first block of this book is dedicated to introducing diverse leading marketing academics’ reflections on the potential of Soft Computing and other AIbased methods for the marketing domain. Following these essays, the book is structured in five main parts, in order to articulate in a more congruent manner the rest of the chapters. In this regard, the reader should be aware of the fact that some of the chapters could be reasonably assigned to more than one part, though they have been finally grouped as follows. The first part deals with segmentation and targeting. Duran et al. analyze the use of different clustering techniques such as k-means, fuzzy c-means, genetic kmeans and neural-gas algorithms to identify common characteristics and segment customers. Next, Markic and Tomic investigate the integration of crisp and fuzzy clustering techniques with knowledge-based expert systems for customer segmentation. Thirdly, Van der Putten and Kok develop predictive data mining for behavioral targeting by data fusion and analyze different techniques such as neural networks, linear regression, k-nearest neighbor and naive Bayes to deal with targeting. Finally, Bruckhaus reviews collective intelligent techniques which allow marketing managers to discover and approach behaviors, preferences and ideas of groups of people. These techniques are useful for new insights into firms’ customer portfolios so they can be better identified and targeted. The second part contains several contributions grouped around marketing modeling. Bhattacharyya explores the use of multi-objective genetic programming to derive predictive models from a marketing-related dataset. Orriols-Puig et al. propose an unsupervised genetic learning approach based on fuzzy association rules to extract causal patterns from consumer behavior databases. Finally, Pereira Preface IX and Tettamanzi introduce a distributed evolutionary algorithm to optimize fuzzy rule-based predictive models of various types of customer behavior. Next, there are two parts devoted to elements of the marketing-mix, specifically applications and solutions for Communication and Product policies. In the third part, Hsu et al. show how a fuzzy analytic hierarchy process helps to reduce imprecision and improve judgment when evaluating the preference of customer opinions about customer relationship management. López and López propose a distributed intelligent system based on multi-agent systems, an analytic hierarchy process and fuzzy c-means to analyze customers’ preferences for direct marketing. Wong also addresses direct marketing but using evolutionary algorithms that describe Bayesian networks from incomplete databases. The fourth part consists of two chapters directly related to Product policy, plus a third dealing with a problem of consumer’s choice based on diverse criteria, mainly functional characteristics of products, though this contribution also has implications for strategic and other marketing-mix areas. Genetic algorithms have proved to be effective in optimizing product line design, according to both Tsafarakis-Matsatsinis and Balakrishnan et al. in their chapters. A dynamic programming algorithm is also used in the second case to seed the genetic algorithm with promising initial solutions. In Beynon et al.’s chapter, probabilistic reasoning is hybridized with analytic hierarchy processes to approach the problem of consumer judgment and the grouping of the preference criteria that drive their product/brand choices. The final part is a set of contributions grouped under e-commerce applications. Sun et al. propose a multiagent system based on case-based reasoning and fuzzy logic for web service composition and recommendation. Dass et al. investigate the use of functional data analysis for the dynamic forecasting of price prediction in simultaneous online auctions. Finally, Beynon and Page deploy probabilistic reasoning and differential evolution to deal with incomplete data for measuring consumer web purchasing attitudes. This book is useful for technicians who apply intelligent systems to marketing, as well as for those marketing academics and professionals interested in the application of advanced intelligent systems. Synthetically, it is especially recommended for the following groups: • • • • Computer Science engineers working on intelligent systems applications, especially Soft-Computing-based Intelligent Systems. Marketers and business managers of firms working with complex information systems. Computer Science and Marketing academics, in particular those investigating synergies between the AI and Marketing. PhD students studying intelligent systems applications and advanced analytical methods for marketing. X Preface Finally, we wish to thank Springer and in particular Prof. J. Kacprzyk, for having given us the opportunity to make real this fascinating and challenging dream. We are also honored and privileged to have received help and encouragement from several notable world marketing academics; we thank you for your support, smart ideas and thoughts. Likewise, we offer our most sincere acknowledgment and gratitude to all the contributors for their rigor and generosity in producing such high quality papers. Last but not least, we especially thank the team of reviewers for their great work. March 2010 Granada (Spain) Jorge Casillas and Francisco J. Martínez-López University of Granada, Spain Contents Essays Marketing and Artificial Intelligence: Great Opportunities, Reluctant Partners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Berend Wierenga 1 Data Mining and Scientific Knowledge: Some Cautions for Scholarly Researchers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nick Lee, Gordon Greenley 9 Observations on Soft Computing in Marketing . . . . . . . . . . . . . . . David W. Stewart 17 Soft Computing Methods in Marketing: Phenomena and Management Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . John Roberts 21 User-Generated Content: The “Voice of the Customer” in the 21st Century . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Eric T. Bradlow 27 Fuzzy Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dawn Iacobucci KDD: Applying in Marketing Practice Using Point of Sale Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adilson Borges, Barry J. Babin Marketing – Sales Interface and the Role of KDD . . . . . . . . . . . Greg W. Marshall 31 35 43 XII Contents Segmentation and Targeting Applying Soft Cluster Analysis Techniques to Customer Interaction Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Randall E. Duran, Li Zhang, Tom Hayhurst Marketing Intelligent System for Customer Segmentation . . . Brano Markic, Drazena Tomic 49 79 Using Data Fusion to Enrich Customer Databases with Survey Data for Database Marketing . . . . . . . . . . . . . . . . . . . . . . . . 113 Peter van der Putten, Joost N. Kok Collective Intelligence in Marketing . . . . . . . . . . . . . . . . . . . . . . . . . 131 Tilmann Bruckhaus Marketing Modelling Predictive Modeling on Multiple Marketing Objectives Using Evolutionary Computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 Siddhartha Bhattacharyya Automatic Discovery of Potential Causal Structures in Marketing Databases Based on Fuzzy Association Rules . . . . . 181 Albert Orriols-Puig, Jorge Casillas, Francisco J. Martı́nez-López Fuzzy–Evolutionary Modeling of Customer Behavior for Business Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 Célia da Costa Pereira, Andrea G.B. Tettamanzi Communication/Direct Marketing An Evaluation Model for Selecting Integrated Marketing Communication Strategies for Customer Relationship Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227 Tsuen-Ho Hsu, Yen-Ting Helena Chiu, Jia-Wei Tang Direct Marketing Based on a Distributed Intelligent System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255 Virgilio López Morales, Omar López Ortega Direct Marketing Modeling Using Evolutionary Bayesian Network Learning Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 Man Leung Wong Contents XIII Product Designing Optimal Products: Algorithms and Systems . . . . . . . 295 Stelios Tsafarakis, Nikolaos Matsatsinis PRODLINE: Architecture of an Artificial Intelligence Based Marketing Decision Support System for PRODuct LINE Designs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337 P.V. (Sundar) Balakrishnan, Varghese S. Jacob, Hao Xia A Dempster-Shafer Theory Based Exposition of Probabilistic Reasoning in Consumer Choice . . . . . . . . . . . . . . . . 365 Malcolm J. Beynon, Luiz Moutinho, Cleopatra Veloutsou E-Commerce Decision Making in Multiagent Web Services Based on Soft Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389 Zhaohao Sun, Minhong Wang, Dong Dong Dynamic Price Forecasting in Simultaneous Online Art Auctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417 Mayukh Dass, Wolfgang Jank, Galit Shmueli Analysing Incomplete Consumer Web Data Using the Classification and Ranking Belief Simplex (Probabilistic Reasoning and Evolutionary Computation) . . . . . . . . . . . . . . . . . . 447 Malcolm J. Beynon, Kelly Page Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475 Marketing and Artificial Intelligence: Great Opportunities, Reluctant Partners Berend Wierenga Professor of Marketing Rotterdam School of Management, Erasmus University e-mail: bwierenga@rsm.nl 1 Introduction Marketing managers make decisions about products, brands, advertising, promotions, price, and distribution channels, based on deep knowledge about customers. The outcomes of marketing decisions are dependent on the behavior of other actors such as competitors, suppliers and resellers. Furthermore, uncertain factors such as the overall economy, the state of the financial sector and (international) political developments play an important role. Marketing decision making not only refers to tactical marketing mix instruments (the well-known 4Ps), but also to strategic issues, such as product development and innovation and long term decisions with respect to positioning, segmentation, expansion, and growth. This short description illustrates that marketing is a complex field of decision making. Some marketing problems are relatively well-structured (especially the more tactical marketing mix problems), but there are also many weakly-structured or even ill-structured problems. Many marketing phenomena can be expressed in numbers, for example sales (in units or dollars), market share, price, advertising expenditures, number of resellers, retention/churn, customer value, etc. Such variables can be computed and their mutual relationships can be quantified. However, there are also many qualitative problems in marketing, especially the more strategic ones. Therefore, besides computation, marketing decision making also involves a large degree of judgment and intuition in which the knowledge, expertise, and experience of professionals play an important role. It is clear that marketing decision making is a combination of analysis and judgment. As we will see below, the analytical part of marketing decision making is well served with a rich collection of sophisticated mathematical models and procedures for estimation and optimization that support marketing decision making. However, this is much less the case for the judgmental part where knowledge and expertise play an important role. The question is whether the acquisition and use of knowledge and expertise by marketing decision makers and their application to actual marketing problems can also benefit from appropriate decision support technologies. In this book on marketing intelligent systems, it is logical to ask what the field of Artificial Intelligence can contribute here. Artificial Intelligence (AI) deals J. Casillas & F.J. Martínez-López (Eds.): Marketing Intelligence Systems, STUDFUZZ 258, pp. 1–8. © Springer-Verlag Berlin Heidelberg 2010 springerlink.com 2 B. Wierenga with human intelligence and how this can be represented in computers. Important topics in AI are knowledge, knowledge representation, reasoning, learning, expertise, heuristic search, and pattern recognition. All these elements are relevant in the daily life of marketing decision makers who constantly use their knowledge, expertise and intuition to solve marketing problems. Therefore, potentially AI can make an important contribution to marketing decision making. However, so far this potential has only been realized to a very limited extent. This contribution takes a closer look at the opportunities for AI in marketing, takes stock of what has been achieved so far, and discusses perspectives for the future. 2 Marketing Problem-Solving Modes We start with a discussion about marketing problem-solving modes. These are specific ways of making marketing decisions. Basically, decision making is dependent on three factors: the marketing problem, the decision maker, and the decision environment. This results in four different marketing problem-solving modes: Optimizing, Reasoning, Analogizing, and Creating (ORAC) (Wierenga and Van Bruggen 1997; 2000). The ORAC model is depicted in Figure 1 and shows the full continuum of how marketing decision makers deal with problems. At the one extreme we have hard calculation (“clocks of mind”), and at the other we have free flows of thought, mental processes without a clear goal (“clouds of mind’). We briefly discuss the four marketing problem-solving modes. O = Optimizing R = Reasoning A = Analogizing C = Creating Clocks of Mind ↑ . . . ↓ Clouds of Mind Fig. 1 The ORAC model of marketing problem-solving modes (Wierenga and van Bruggen 1997; 2000) Optimizing implies that there is an objectively best solution that can be reached by proper use of the marketing instruments. This is only possible if we have precise insight in the mechanism behind the variable that we want to optimize (e.g. sales, market share or profit). Once this mechanism is captured in a mathematical model, the best values for the marketing instruments (dependent on the objective function) can be found by applying optimization or simulation. An example of optimizing is deciding on the best media plan (i.e. the allocation over media such as TV, press, internet) for an advertising campaign, once the advertising budget and the relevant reach and costs data of the media are known. Marketing and Artificial Intelligence: Great Opportunities, Reluctant Partners 3 Reasoning means that a marketer has a representation (mental model) of certain marketing phenomena in mind, and uses this as a basis for making inferences and drawing conclusions. For example, the decision maker may have a mental model of the factors that determine the market share of his brand. Suppose that this is relatively low in one particular geographical area. The manager might then reason (if ….then…) that this can be due to several possible causes, (i) deviant preferences of consumers; (ii) low efforts of salespeople; or (iii) relatively strong competition (Goldstein 2001). Market research can help to verify each of these possible causes and will result in decisions about possible actions (e.g. change the taste of the product; increase the salesforce). The outcomes of market research may also lead to the updating of managers’ mental models. Analogizing takes place when marketing decision makers, confronted with a problem, recall a similar problem that previously occurred and was solved in a satisfactory way. Decision makers often organize their experiences in the form of “stories”. New cases are easily interpreted using existing stories, and solutions are found quickly, often automatically. This type of analogical reasoning occurs very often in marketing. For example, when a new product is introduced, experiences with earlier product introductions act as points of reference. Creating occurs when the decision maker searches for novel and effective ideas and solutions. This means mapping and exploring the problem’s conceptual space and involves divergent thinking. In marketing, creating is a very important marketing problem-solving mode. Marketers are always looking for innovative product ideas, catchy advertising themes and imaginative sales promotion campaigns. 3 Marketing Problem-Solving Modes and Decision Support Technologies Over time a rich collection of decision aids have become available that can support marketing managers to improve the effectiveness and efficiency of their decisions. The complete set is referred to as marketing management support systems (MMSS). (Wierenga and van Bruggen 2000). Figure 2 shows how the decision support technologies used in these MMSS are related to the marketing problemsolving modes. The mapping to marketing problem-solving modes is not exactly one-to-one, but Figure 2 shows the overall tendencies. Marketing management support systems can be divided in two categories, datadriven and knowledge-driven. Marketing data have become available abundantly over the last few decades (e.g. scanner data; internet data) and data-driven decision support technologies are very prominent in marketing. They are particularly important for optimizing and reasoning. Methods from operations research (OR) and econometrics play an important role here. For example, OR methods can be used to optimally allocate the advertising budget over advertising media and econometric analysis can help to statistically determine the factors that affect market share. As we have just seen, the latter information is useful for reasoning about possible marketing actions, and for the updating of managers’ mental models. 4 B. Wierenga Marketing ProblemSolving Modes Optimizing Decision Support Technologies Data-driven • • • Operations Research (OR) Econometric Modeling Predictive Modeling/NN Reasoning Knowledge-driven • Analogizing • • Knowledge-Based Systems/ Expert Systems Analogical Reasoning/ Case-Based Reasoning Creativity Support Systems Creating Fig. 2 Marketing problem-solving modes and decision support technologies Predictive modeling techniques used in Customer Relationship Management (CRM) and direct marketing are also data-driven. (Neural nets-NN is a predictive modeling technique that has its roots in AI). Knowledge-driven decision support technologies are particularly useful for marketing problem-solving modes that deal with weakly structured problems, parts (i.e. the qualitative element) of reasoning, analogizing, and creating. Knowledge-based systems (KBS) and expert systems (ES) are important examples. The latter, in particular, can also be used for reasoning about the factors behind particular marketing phenomena, for example the success of new products, or the effect of an advertising campaign. Decision support technologies based on analogical reasoning, such as case-based reasoning (CBR) have great potential for the analogizing and creating modes. This is also a potential application area for creativity support systems (Garfield 2008). 4 The State of Artificial Intelligence (AI) in Marketing Figure 2 shows that the potential for knowledge-driven decision support technologies in marketing is high. Contributions from AI are possible for three of the four marketing problem-solving modes. However, reality does not reflect this. To date, data-driven approaches, mostly a combination of operations research and econometric methods are dominant in marketing management support systems. It is safe to say that data-driven, quantitative models (i.e. the upper-right corner of Figure 2) make up over 80% of all the work in decision support systems for marketing at Marketing and Artificial Intelligence: Great Opportunities, Reluctant Partners 5 this moment. Compared to this, the role of artificial intelligence in marketing is minor1. The number of publications about AI approaches in marketing literature is limited and the same holds true for the presence of marketing in AI literature. In 1958 Simon and Newell wrote that “the very core of managerial activity is the exercise of judgment and intuition” and that “large areas of managerial activity have hardly been touched by operations and management science”. In the same paper (in Operations Research) they foresaw the day that it would be possible “to handle with appropriate analytical tools the problems that we now tackle with judgment and guess”. Strangely enough, it does not seem that judgment and intuition in marketing have benefitted a lot from the progress in AI since the late fifties. It is true that AI techniques are used in marketing (as we will see below), but only to a limited degree. There are several (possible) reasons for the limited use of AI in marketing. • Modern marketing management as a field emerged in the late 1950s. At that time, operations research and econometrics were already established fields. In fact, they played a key role in the development of the area of marketing models (Wierenga 2008), which is one of the three academic pillars of contemporary marketing (the other pillars are consumer behavior and managerial marketing). Artificial intelligence as a field was only just emerging at that time. • OR and econometrics are fields with well-defined sets of techniques and algorithms, with clear purposes and application goals. They mostly come with userfriendly computer programs that marketers can directly implement for their problems. However, AI is a heterogeneous, maybe even eclectic, set of approaches, which often takes considerable effort to implement. Moreover, most marketing academics are not trained in the concepts and theories of AI. • The results of applications of OR and econometrics can usually be quantified, for example as the increase in number of sales or in dollars of profit. AI techniques, however, are mostly applied to weakly-structured problems and it is often difficult to measure how much better a solution is due to the use of AI, for example a new product design or a new advertising campaign. Marketers seem to be better at ease with rigorous results than with soft computing. There may also be reasons on the side of AI. • There seems to be little attention for marketing problems in AI. A recent poster of the “The AI Landscape” (Leake 2008) shows many (potential) applications of AI, ranging from education, logistics, surgery, security, to art, music, and entertainment, but fails to mention marketing, advertising, selling, promotions or other marketing-related fields. 1 Here we refer to the explicit use of AI in marketing. Of course, AI principles may be imbedded in marketing-related procedures such as search algorithms for the Internet). 6 B. Wierenga • Perhaps the progress in AI has been less than was foreseen in 1958. In general, there has been a tendency of over-optimism in AI (a point in case is prediction about when a computer would be the world’s chess champion). The promised analytical tools to tackle judgmental marketing problems may come later than expected. 4.1 Applications of AI in Marketing The main applications of AI in marketing so far are expert systems, neural nets, and case-based reasoning. We discuss them briefly. 4.1.1 Expert Systems In the late eighties, marketing knowledge emerged as a major topic, together with the notion that it can be captured and subsequently applied by using knowledgebased systems. In marketing, this created a wave of interest in expert systems. They were developed for several domains of marketing (McCann and Gallagher 1990). For example: systems (i) to find the most suitable type of sales promotion; (ii) to recommend the execution of advertisements (positioning, message, presenter); (iii) to screen new product ideas, and (iv) to automate the interpretation of scanner data, including writing reports. Around that time, over twenty expert systems were published in marketing literature (Wierenga & van Bruggen 2000 Chapter 5).An example of a system specially developed for a particular marketing function is BRANDFRAME (developed by Wierenga, Dalebout, and Dutta 2000; see also Wierenga and van Bruggen 2001). This system supports a brand manager, which is a typical marketing job. In BRANDFRAME the situation of the (focal) brand is specified in terms of its attributes, competing brands, retail channels, targets and budgets. When new marketing information comes in, for example from panel data companies such as Nielsen and IRI, BRANDFRAME analyzes this data and recommends the marketing mix instruments (for example: lower the price; start a sales promotion campaign). It is also possible to design marketing programs in BRANDFRAME, for example for advertising or sales promotion campaigns. The system uses frame-based knowledge representation, combined with a rulebased reasoning system. In recent years, marketing literature has reported few further developments in marketing expert systems. 4.1.2 Neural Networks and Predictive Modeling Around 2000, customer relationship management (CRM) became an important topic in marketing. An essential element of CRM (which is closely related to direct marketing) is the customer database which contains information about each individual customer. This information may refer to socio-economic characteristics (age, gender, education, income), earlier interactions with the customer (e.g. offers made and responses to these offers, complaints, service), and information about the purchase history of the customer (i.e. how much purchased and when). This data can be used to predict the response of customers to a new offer or to predict customer retention/churn. Such Marketing and Artificial Intelligence: Great Opportunities, Reluctant Partners 7 predictions are very useful, for example for selecting the most promising prospects for a mailing or for selecting customers in need of special attention because they have a high likelihood of leaving the company. A large set of techniques is available for predictive modeling. Prominent techniques are neural networks (NN) and classification and regression trees (CART), both with their roots in artificial intelligence. However, also more classical statistical techniques are used such as discriminant analysis and (logit) regression (Malthouse and Blattberg 2005; Neslin et al 2006). CRM is a quickly growing area of marketing. Companies want to achieve maximum return on the often huge investments in customer databases. Therefore, further sophistication of predictive modeling techniques for future customer behavior is very important. Fortunately, this volume contains several contributions on this topic. 4.1.3 Analogical Reasoning and Case-Based Reasoning (CBR) Analogical reasoning plays an important role in human perception and decision making. When confronted with a new problem, people seek similarities with earlier situations and use previous solutions as the starting point for dealing with the problem at hand. This is especially the case in weakly structured areas, where there is no clear set of variables that explain the relevant phenomena or define a precise objective. In marketing we have many such problems, for example in product development, sales promotions, and advertising. Goldstein (2001) found that product managers organize what they learn from analyzing scanner data into a set of stories about brands and their environments. Analogical reasoning is also the principle behind the field of case-based reasoning (CBR) in Artificial Intelligence. A CBR system comprises a set of previous cases from the domain under study and a set of search criteria for retrieving cases for situations that are similar (or analogous) to the target problem. Applications of CBR can be found in domains such as architecture, engineering, law, and medicine. By their nature, many marketing problems have a perfect fit with CBR. Several applications have already emerged, for example CBR systems for promotion planning and for forecasting retail sales (see Wierenga & van Bruggen 2000, Chapter 6). A recent application uses CBR as a decision support technology for designing creative sales promotion campaigns (Althuizen and Wierenga 2009). We believe that analogical reasoning is a fruitful area for synergy between marketing and AI. 4.2 Perspective Although there is some adoption of AI approaches in marketing, the two areas are almost completely disjoint. This is surprising and also a shame, because the nature of many marketing problems makes them very suitable for AI techniques. There is a real need for decision technologies that support the solution of weakly-structured marketing problems. Van Bruggen and Wierenga (2001) found that most of the existing MMSS support the marketing problem-solving mode of optimizing, but that they are often applied in decision situations for which they are less suitable (i.e. where the marketing problem-solving modes of reasoning, analogizing orcreating are applicable). Their study also showed that a bad fit between the 8 B. Wierenga marketing-problem-solving mode and the applied decision support technology results in significantly less impact of the support system. It would be fortunate if further progress in AI can help marketing to deal with the more judgmental problems of its field. Reducing the distance between marketing and AI also has an important pay-off for AI. Marketing is a unique combination of quantitative and qualitative problems, which gives AI the opportunity to demonstrate its power in areas where operations research and econometrics cannot reach. Marketing is also a field where innovation and creativity play an important role. This should appeal to the imaginative AI people. Hopefully the current volume will be instrumental in bringing marketing and AI closer together. References Althuizen, N.A.P., Wierenga, B.: Deploying Analogical Reasoning as a Decision Support Technology for Creatively Solving Managerial Design Problems. Working paper. Rotterdam School of Management, Erasmus University (2009) Garfield, M.J.: Creativity Support Systems. In: Burnstein, F., Holsapple, C.W. (eds.) Handbook on Decision Support Systems. Variations, vol. 2, pp. 745–758. Springer, New York (2008) Goldstein, D.K.: Product Manager’s Use of Scanner Data: a Story of Organizational Learning. In: Desphandé, R. (ed.) Using Market Knowledge, pp. 191–216. Sage, Thousand Oaks (2001) Leake, D.B.: AI Magazine Poster: The AI Landscape. AI Magazine 29(2), 3 Malthouse, E.C., Blattberg, R.C.: Can we predict customer lifetime value? Journal of Interactive Marketing 19(1), 2–16 (2005) Mc Cann, J.M., Gallagher, J.P.: Expert Systems for Scanner Data Environments. Kluwer Academic Publishers, Boston (1990) Neslin, S.A., Gupta, S., Kamakura, W., Lu, J., Mason, C.H.: Defection Detection: Measuring and Understanding the Predictive Accuracy of Customer Churn Models. Journal of Marketing Research 43, 204–211 (2006) Simon, H.A., Newell, A.: Heuristic Problem Solving: the Next Advance in Operations Research. Operations Research 6, 1–10 (1958) Van Bruggen, G.H., Wierenga, B.: Matching Management Support Systems and Managerial Problem-Solving Modes: The Key to Effective Decision Support. European Management Journal 19(3), 228–238 (2001) Wierenga, B. (ed.): Handbook of Marketing Decision Models, p. 630. Springer Science + Business Media, New York (2008) Wierenga, B., van Bruggen, G.H.: The Integration of Marketing Problem-Solving Modes and Marketing Management Support Systems. Journal of Marketing 61(3), 21–37 (1997) Wierenga, B., van Bruggen, G.H.: Marketing Management Support Systems: Principles, Tools, and Implementaiton, p. 341. Kluwer Academic Publishers, Boston (2000) Wierenga, B., Van Bruggen, G.H.: Developing a Customized Decision Support System for Brand Managers. Interfaces 31(3) Part 2(2), 128–145 (2001) Wierenga, B., Dalebout, A., Dutta, S.: BRANDFRAME: A Marketing Management Support System for the Brand Manager. In: Wierenga, B., van Bruggen, G. (eds.) Marketing Management Support Systems: Principles, Tools, and Implementation, pp. 231–262. Kluwer Academic Publishers, Boston (2000) Data Mining and Scientific Knowledge: Some Cautions for Scholarly Researchers Nick Lee1 and Gordon Greenley2 1 Professor of Marketing and Organizational Research and Marketing Research Group Convenor Aston Business School, Birmingham, UK Co-Editor: European Journal of Marketing 2 Professor of Marketing and Marketing Subject Group Convenor Aston Business School, Birmingham, UK Co-Editor: European Journal of Marketing 1 Introduction Recent years have seen the emergence of data analytic techniques requiring for their practical use previously unimaginable raw computational power. Such techniques include neural network analysis, genetic algorithms, classification and regression trees, v-fold cross-validation clustering and suchlike. Many of these methods are what could be called ‘learning’ algorithms, which can be used for prediction, classification, association, and clustering of data based on previouslyestimated features of a data set. In other words, they are ‘trained’ on a data set with both predictors and target variables, and the model estimated is then used on future data which does not contain measured values of the target variable. Or in clustering methods, an iterative algorithm looks to generate clusters which are as homogenous within and as heterogeneous between as possible. Such analytic methods can be used on data collected with the express purposes of testing hypotheses. However, it is when such methods are employed on large sets of data, without a priori theoretical hypotheses or expectations, that they are known as data mining. In fact, it appears that such is the explosion in use of such methods, and in particular their use in commercial contexts such as customer relationship management or consumer profiling, that it is the methods themselves which are considered to be ‘data mining’ methods. However, it should be made clear at the outset of this essay that it is the use that they are put to which should be termed ‘data mining’, not the tools themselves (Larose 2005). This is despite the naming of software packages like ‘Statistica Data Miner’, which sell for sums at the higher end of 6-figures to commercial operations. In fact, a technique as ubiquitous as multiple regression can be used as a data mining tool if one wishes. It is the aim of this essay to place the recent exponential growth of the use of data mining methods into the context of scientific marketing and business research, and in particular to sound a note of caution for social scientific researchers about the over-use of a data-mining approach. In doing so, the fundamental nature J. Casillas & F.J. Martínez-López (Eds.): Marketing Intelligence Systems, STUDFUZZ 258, pp. 9–15. © Springer-Verlag Berlin Heidelberg 2010 springerlink.com 10 N. Lee and G. Greenley of data mining is briefly outlined. Following this, data mining is discussed within a framework of scientific knowledge development and epistemology. Finally, the potential use of data mining in certain contexts is noted. We will conclude with some important points that business and marketing scholars should consider when considering the use of data mining approaches. 2 The Data Mining Method Data mining is one part of a wider methodology termed Knowledge Discovery in Databases (KDD). Within this process, the term data mining refers to the uncovering of new and unsuspected relationships in, and the discovery of new and useful knowledge from, databases (e.g. Adriaans and Zantinge, 1996; Hand et al, 2001). While it should be natural for the scholar to immediately consider the question of what exactly is knowledge, this will be dealt with in the next section. In a more practical context, scientists and businesspeople deal with large databases on a dayto-day basis. In many cases, they use the data to answer questions that they pose in a very structured way – such as ‘what is the difference between the mean level of job satisfaction across high and low-stress salespeople’ or ‘which customers bought brand x during August’. Such structured queries have been common practice for many years. The difference between the data mining approach and a normal structured interrogation of a data set is that, when data mining, one starts without such a structured question, but instead is interested in exploring the database for any potential ‘nuggets’ of interest. Another key point of interest is that – while this is not an essential component of a data-mining approach – most methods of data mining involve learning algorithms. Unlike traditional analysis of data, learning algorithms (such as genetic algorithms or neural networks) are able to be ‘trained’ to create rules which are able to describe a data set, that are then able to work on new data. While humans could of course train themselves to do this, the advantage of the learning algorithm is that it can work with far larger data sets, in far less time, than humans – as long as the data set contains at least some structure. 3 Data Mining and Scientific Knowledge The characteristics of the data mining approach introduced above have significant relevance to its use to generate scientific knowledge. Of course, as shall be seen subsequently, data mining has use in many contexts outside of science as well. However, as Editors of the European Journal of Marketing, a major journal dedicated to advances in marketing theory, it is their use in scientific knowledge development in marketing (and by extension more general business or social research) which is our primary concern in this essay1. Marketing has long debated 1 It is important to note that – while we were invited to write this essay as Editors of EJM – the opinions expressed here should not be taken to represent a formal editorial policy or direction for EJM in any manner. Data Mining and Scientific Knowledge: Some Cautions for Scholarly Researchers 11 its status as a science (e.g. Buzzell, 1963, Hunt, 1983; 1990; 2003, Brown, 1995), with various scholars taking different viewpoints on both the nature of science itself, and whether marketing can be considered to be a science. Hunt’s (e.g. 1983) work is arguably the most articulate and significant corpus regarding this issue, and it is defensible to say that – working with the premise that one wants to class marketing as a science – Hunt’s delineation of the nature of science (e.g. 1983) can be taken as broadly coherent in the context of marketing research. Hunt defines three essential characteristics of a science (1983: pp. 18); “(1) a distinct subject matter, (2) the description and classification of the subject matter, and (3) the presumption that underlying the subject matter are uniformities and regularities which science seeks to discover”. Hunt also adds (pp. 18-19) that to be termed a science, a discipline must employ the scientific method; which he defines as a “set of procedures”. Like the nature of science itself, the scientific method has been subject to considerable debate and controversy over the last century (e.g. Feyerabend, 1993). One of the key areas of misunderstanding is whether the method refers to the practical techniques used for discovery, or the conceptual/theoretical method used to justify a discovery as knowledge (Lee and Lings 2008). Hunt (1983) points out that the scientific method is not dependent on the use of particular data collection methods, tools, or analysis techniques, since it is of course the case that different sciences use different tools as appropriate. Instead, the scientific method should more accurately be perceived as a method for justifying the knowledge claims uncovered by investigation (Lee and Lings 2008). In this sense, there are multiple (perhaps infinite) ways of discovery, and of making knowledge claims about the world, but at present only one scientific method of justifying those claims as actual knowledge. This method – termed more formally the hypothetico-deductive method – has proven to be the foundation of scientific research since its formal articulation by Karl Popper. Thus, in exploring the usefulness of data mining for scientific research, it is naturally necessary to do so in relation to where it may sit within a hypothetico-deductive approach to research. While a full explication of the hypothetico-deductive method is outside the scope of this short essay, it is the term deductive which is of most relevance to the present discussion. Of course, deduction refers to the creation of theory from logical argument, which may then be tested through empirical observation. While it is often characterized as a cycle of induction and deduction, the essence of the hypothetico-deductive method is the idea that one should create theoretical hypotheses through deductive reasoning, and then collect empirical data in an attempt to falsify those hypotheses. Certainly, one may begin the cycle by using inductive reasoning from some empirical observation or discovery, but until formal hypotheses are generated and subsequently tested, one should not claim to have created any scientific knowledge. The idea of falsification is of critical importance in this context. Current definitions of the nature of scientific research depend on the assumption that empirical data alone can never prove a hypothesis, but only disprove it. Thus, the hypothetico-deductive method can be seen as a way of systemizing the search for falsifying evidence about our hypotheses. This is in direct opposition to the pure empiricist or logical positivist position which was heretofore dominant in 12 N. Lee and G. Greenley scientific research. Such approaches instead considered empirical observations not just to be sufficient proof alone, but in fact that all other types of evidence (e.g. rational thought and logical deduction) were of no use in knowledge generation. If one considers the hypothetico-deductive method to be the foundation of scientific research within marketing (cf. Hunt 1983), then the place of data mining is worthy of some discussion. Drawing from the nature of data mining as defined above, it would seem that data mining may have some use in a scientific knowledge creation process, but that this use would be limited. More specifically, the data mining approach is fundamentally an inductive one, in which a data set is interrogated in an exploratory fashion, in the hope of turning up something of interest. Surely, if one is working within a hypothetico-deductive paradigm of knowledge generation, any findings from a purely data mining study could never be considered as actual knowledge. Instead, such findings should be treated as knowledge claims, and used to help generate explicit theoretical hypotheses which can then be tested in newly-designed empirical studies, which collect new data. Only when these theoretical hypotheses fail to be falsified with additional empirical work can the knowledge claim then be considered as knowledge. It is certainly the case that the use of theory to help explain these empirical discoveries is also worthy of significant discussion. Or in other words whether purely empirical results from a data mining study only are enough to justify hypothesis generation and further testing. However, a full discussion of this is outside the present scope, given the short space available. Even so, our short answer to this question would be that the emergent inductive empirical relations would need theoretical deductive explanation as well, in order to justify them as testable hypotheses in a scientific context. In this sense, empirical data mining results are but a single strand of evidence or justification for a hypothesis, rather than sufficient alone. It is important to make clear however that this position refers to the data mining method, not to any particular technique or algorithm. Certainly, many algorithms commonly of use in data mining applications can and have been usefully employed in a deductive manner in scientific studies – such as multiple regression, principle components analysis, clustering, classification trees, and the like. However, the critical issue is not one of technique, but of the underlying epistemological position of the task employing the technique. 4 Data Mining in a Practical Context Notwithstanding the above, it is not the intention of this essay to decry the use of data mining approaches in general, since they are clearly of major potential use in both commercial and some scientific applications. Beginning with commercial applications, it is clear that marketing-focused firms can employ data mining methods to interrogate the huge customer databases which they routinely generate. Such work is common, and can include such tasks as market segmentation, customer profiling, and auditing. For example, it is well-known that Google utilizes data mining methods to predict which advertisements are best matched to which websites. Thus, without any actual knowledge (as we would term it) of why, Google can predict an advertisement’s likely success depending on how it is Data Mining and Scientific Knowledge: Some Cautions for Scholarly Researchers 13 matched (Anderson 2008). Considering the terabytes of data Google collects constantly, such methods are likely to be the most effective way to predict success. Yet the question of whether raw prediction is actually scientific knowledge is moot in this and most other practical situations. As most business researchers know, few business organizations are particularly interested in explaining the theory of why things are related, but only in predicting what will happen if variables are changed. In other words, what ‘levers’ can be manipulated to improve performance? Data mining is an ideal tool for this task. However, raw data mining is also of significant use in many scientific fields outside of the business or social sciences. For example, sciences such as biochemistry work with massive data sets in many cases. In these situations, data mining can be usefully employed in uncovering relationships between for example genetic polymorphisms and the prevalence of disease. There is a significant difference between such scientific work and a typical business research study. In such biosciences, researchers often work within a very exploratory, or descriptive, context, and they also often work within contexts without large amounts of competing or unmeasured variables. For example, if one is working within a database of the human genome, then this is all the data. Conversely, if one is working within a database of customer characteristics, there may be many hundreds of unmeasured variables of interest, and any description of that database will be able to incorporate but a tiny subset of the possible explanatory variables. Even so – as is the case within neuroscientific research at present – purely exploratory or descriptive approaches (which data mining is useful for) must eventually be superseded by theory-driven hypothetico-deductive approaches (e.g. Senior and Russell, 2000). 5 Discussion and Conclusions The aim of this invited essay was to explore the implications and uses of data mining in the context of scientific knowledge generation for marketing and business research. In doing so, we defined both data mining and scientific knowledge. Importantly, data mining was defined as a method of exploration, not as a set of particular tools or algorithms. Knowledge was defined as distinct from a knowledge claim, in that a knowledge claim had not been subject to a hypothetico-deductive attempt at falsification. The importance of this distinction is that in most cases one cannot claim data mining approaches as tools of knowledge generation in a scientific context. At best, they are highly useful for the generation of hypotheses from data sets, which may previously have been unexplored. In this way, it is interesting to draw parallels with qualitative research approaches such as grounded theory (e.g. Glaser and Strauss, 1967). Glaser’s approach to grounded theory instructs that no appreciation of prior theories should be made before either collecting or analyzing data, in order to ‘let the data speak’ (Glaser 1992). Any argument in favor of data mining as a knowledge generation tool must therefore look to such approaches as justification. However, it is our view that – while such methods can result in truly original findings which would be unlikely to emerge from any other method – those findings should always be considered preliminary knowledge claims until further confirmatory testing. 14 N. Lee and G. Greenley This is because, without a priori theoretical expectations (i.e. hypotheses), one is always at risk of over-interpreting the data. In fact, many data mining techniques use the term ‘overfitting’ to refer to this situation (Larose, 2005). In such an instance, one’s findings are essentially an artifact of the data set, and may not bear relation to the rest of the world. In other words, the training set is explained increasingly exactly, but the results are increasingly less generalizable to other data. Of course, if your data set is all of the relevant data in the world (as is the case in some scientific contexts), this is not a problem. However in most situations, and particularly within the business and social research contexts, our data contains only a subset of the available data, in terms of both subjects and possible variables. Overfitting in this case results in findings which are likely to have low external validity. Thus, we urge business and social researchers to exercise caution in the application of data mining in scientific knowledge generation. Even so, this is not to say that we consider it to be of no use at all. Just as many other exploratory techniques are of use in the hypothetico-deductive cycle, data mining may provide extremely valuable results in the context of the knowledge generation process as a whole. However, researchers would be well advised to avoid presenting the findings of pure data mining as anything other than preliminary or exploratory research (although of course this may be of significant use in many cases). Although we did not specifically discuss it here, we would also urge researchers to make sure they are knowledgeable in the appropriate use of various data mining algorithms, rather than using them as a ‘black box’ between their data and results. Such an approach runs the risk of being characterized as ‘data-driven’ and therefore should be given little time at top-level journals. In this way, we also urge editors and reviewers at journals to think carefully about the actual contribution of such studies, despite their often complex and impressive technical content. In conclusion, it is our view that explanatory theory is the key contribution of scientific research, and this should not be forgotten. Theory allows us to explain situations and contexts beyond our data, in contrast to pure prediction, which may have no real explanatory value whatsoever. While it may be of interest in many situations, it should not be characterized as scientific knowledge. References Adriaans, P., Zantinge, D.: Data Mining. Addison-Wesley, Harlow, England (1996) Anderson, C.: The End of Science: The Data Deluge Makes the Scientific Method Obsolete. In: WIRED, vol. 16 (7), pp. 108–109 (2008) Buzzell, R.D.: Is Marketing a Science? Harvard Business Review 41(1), 32–40 (1963) Brown, S.: Postmodern Marketing, Thompson, London (1995) Feyerabend, P.K.: Against Method, 3rd edn. Verso, London (1993) Glaser, B.G.: Basics of Grounded Theory Analysis. Sociology Press, Mill Valley (1992) Glaser, B.G., Strauss, A.L.: The Discovery of Grounded Theory: Strategies for Qualitative Research. Aldine, Chicago (1967) Hand, D., Mannila, H., Smyth, P.: Principles of Data Mining. MIT Press, Cambridge (2001) Data Mining and Scientific Knowledge: Some Cautions for Scholarly Researchers 15 Hunt, S.D.: Marketing Theory: The Philosophy of Marketing Science. Irwin, Homewood, IL (1983) Hunt, S.D.: Controversy in Marketing Theory: For Reason, Realism, Truth and Objectivity. M.E. Sharpe, Armonk (2003) Hunt, S.D.: Truth in Marketing Theory and Research. Journal of Marketing 54, 1–15 (1990) Lee, N., Lings, I.: Doing Business Research. Sage, London (2008) Senior, C., Russell, T.: Cognitive neuroscience for the 21st century. Trends in Cognitive Science 4, 444–445 (2000) Observations on Soft Computing in Marketing David W. Stewart Dean of and Professor of Management and Marketing, A. Gary Anderson Graduate School of Management, University of California, Riverside, California, USA Marketing managers make use of a variety of computer-based systems to aid decision- making. Some of these models would be considered “hard” models in the sense that they are based on quantitative data, usually historical data related to some type of market response and some empirically derived functional form of the relationships among actions in the market and market response (Hanssens, Parsons and Schultz 2008). Such models have been widely employed to decisions involving pricing and promotion, advertising scheduling and response, product design, and sales call scheduling, among others (Lilien and Rangaswamy 2006). These models, while very useful, require very rich data, as well strong assumptions about the generalizability of historical data to future events. These are assumptions likely to be less and less valid in an increasingly volatile world that includes regular introduction of new means of communications and product/service distribution, as well as new product and service innovations. Quantitative models in marketing are also limited by two other factors. First, it is often not possible to obtain certain types of data that would be desirable for making marketing decision - for example, experimental data on customer response to different product characteristics, advertising levels, product prices, etc. Although some data may be available from interviews, test market experiments, and the like, it is often necessary to supplement them with the judgment of experienced marketing managers. The second limitation is related to the complexity of many marketing factors, many of which are unquantifiable. The decision environment may simply to be too complex to develop a quantitative model that captures all of the relevant parameters. As a result of these limitations marketers have sought to build models that not include hard quantitative components, but also soft models that incorporate managerial judgment. These models are not “expert systems” in the classic sense of the term because they do not capture a set of replicable rules that would be characteristic of the use of artificial intelligence (Giarratano and Riley 2004, Little and Lodish 1981). Rather, decision calculus attempts to capture the subjective judgments, and hence, the experience of a decision maker within the context of a predictive model. For at least four decades the marketing literature has documented the development and commercial use of models that incorporate the judgments of experienced managers. Models have been published which assist managers in making decisions about a wide range of marketing variables, including prices, couponing J. Casillas & F.J. Martínez-López (Eds.): Marketing Intelligence Systems, STUDFUZZ 258, pp. 17–19. © Springer-Verlag Berlin Heidelberg 2010 springerlink.com 18 D.W. Stewart efforts, advertising spending, media selection, and sales force scheduling. Many of these systems require that managerial expertise be used to set parameters and even to specify model forms. The way in which these models use judgmental data, the nature of their construction, and the purposes to which they are put differ in important ways from those of the typical expert system. Montgomery and Weinberg (1973) describe the typical decision calculus modeling exercise as: • • • • Managers first verbalize their implicit model of the situation or issue of interest, specifying factors that influence a criterion variable of interest and the relationships of factors to one another and to the criterion variable; This verbal model is translated to a formal mathematical model. In most applications the response function has two components, a current component and a delayed (or lagged) component. Lilien and Kotler (1983) provide a useful overview of typical forms these models take in marketing applications; Parameters associated with the mathematical model are estimated; and An interactive procedure is implemented that allows managers to examine the influence of variations of particular factors on the criterion. By examining the model outputs obtained by changing model inputs, managers can examine alternative decisions and determine the sensitivity of the criterion to changes in particular input factors; Obviously, the development of a useful decision support tool is a primary benefit of model building involving decision calculus. Little and Lodish (1981) argue that numerous additional benefits also accrue from the model building exercise. Among these additional benefits are: • • • • Model building facilitates the collection of data; It makes assumptions explicit; It identifies areas of disagreement and the nature of that disagreement; and It helps identify needs for information that have a high payoff potential. Decision calculus models have a great deal in common with soft computer, though soft computing clearly brings a broader array of tools and methods to the task of informing decision-making. Soft computing also takes advantage of the enormous increase in computational power and the new techniques in biological computation that have emerged since the development of decision calculus models (Abraham, Das, and Roy 2007). Nevertheless, there is a common philosophical and methodological history that unites these different types of models. The underlying notion is that complex problems can be solved at a molar level as an alternative computational models that seek to fit quantitative models at a more micro level. Although it has demonstrate its utility in a host of venues, soft computing has yet to demonstrate its utility in solving practical marketing problems. It seems only a matter of time before it does so given the complex data sets now available to marketing organizations. It is also likely that these tools will carry benefits similar to those already demonstrated for decision calculus models in marketing. Observations on Soft Computing in Marketing 19 References Abraham, A., Das, S., Roy, S.: Swarm Intelligence Algorithms for Data Clustering. In: Maimon, O., Rokach, L. (eds.) Soft Computing for Knowledge Discovery and Data Mining, pp. 279–313. Springer, New York (2007) Giarratano, J.C., Riley, G.: Expert Systems, Principles and Programming, 4th edn. PWS Publishing, New York (2004) Hanssens, D.M., Parsons, L.J., Schultz, R.L.: Market Response Models: Econometric and Time Series Analysis, 2nd edn. Springer, New York (2008) Lilien, G., Kotler, P.: Marketing Decision Making: A Model-building Approach. Harper and Row, New York (1983) Lilien, G.L., Rangaswamy, A.: Marketing Decision Support Models. In: Grover, R., Vriens, M. (eds.) The Handbook of Marketing Research, pp. 230–254. Sage, Thousand Oaks (2006) Little, J.D.C., Lodish, L.M.: Judgment-based marketing decision models: Problems and possible solutions/commentary on Judgment-Based Marketing Decision Models. Journal of Marketing 45(4), 13–40 (1981) Montgomery, D., Weinberg, C.: Modeling Marketing Phenomena: A Managerial Perspective. Journal of Contemporary Business (Autumn), 17–43 (1973) Soft Computing Methods in Marketing: Phenomena and Management Problems John Roberts Professor of Marketing College of Business and Economics, Australian National University (Australia) and London Business School (UK) e-mail: john.roberts@anu.edu.au 1 Introduction Soft computing techniques gained popularity in the 1990s for highly complex problems in science and engineering (e.g., Jang et al. 1997). Since then, they have slowly been making their way into management disciplines (Mitra et al. 2002). In order to understand the potential of these methods in marketing, it is useful to have a framework with which to analyze how analytical methods can provide insight to marketing problems. Marketing actions Brands and the marketing mix that supports them Customer linking Customer management including acquisition, retention and maximization Marketplace phenomena Customer behavior, including beliefs, needs, preferences and actions Market environment including competition, channels, collaborators, and climate Market feedback Market information: marketing research and intelligence Market analysis and insight Market sensing Fig. 1 A Model of the Market Decision Making Process J. Casillas & F.J. Martínez-López (Eds.): Marketing Intelligence Systems, STUDFUZZ 258, pp. 21–26. © Springer-Verlag Berlin Heidelberg 2010 springerlink.com 22 J. Roberts Marketing may be regarded as harnessing the resources of the organization to address the needs of its target customers, given the marketplace environment in which it competes (the top arrow in Figure 1). George Day calls this process “customer linking” (Day 1994). Actions in the top right box can be analyzed either from an internal perspective in terms of the products and services of the organization and the marketing mix that supports them, or externally in terms of its customers: how it attracts, retains and maximizes the value it provides to and captures from them. However, in order to focus the organization’s actions, an understanding of the environment is necessary, and feedback from the marketplace helps the manager better target her actions to where they will be most effective (the bottom arrows in Figure 1). Day calls this function “market sensing.” Market sensing has the dual elements of gathering data from the market and transforming those data into insights for action, by using suitable analytical tools. Soft computing tools form one weapon in the marketing analyst’s toolkit to provide that insight. In understanding the potential (and limitations) of soft computing tools, it is useful to analyze this environment. This chapter specifically examines the management actions for which the suite of techniques is well-suited, and the phenomena on which it can throw insight (the two top boxes in Figure 1). Details of the techniques of soft computing that belong to the bottom box are covered elsewhere in this volume. 2 Marketplace Phenomena Soft computing has particular strengths in the case of large databases and complex phenomena. To understand where these are most likely to occur it is useful to decompose the consumer decision. One traditional method of analyzing consumer decisions is by use of Lavidge and Steiner (1961)’s Hierarchy of Effects model (also known as the demand funnel and a variety of other names). This model is illustrated in Figure 2: One major driver of complexity of models (in terms of number of observations, parameters, and interactions between variables) is that of heterogeneity. When we have to allow for differences between individual consumers (or a large number of groups of consumers), the tractability of traditional models is likely to come under threat. In marketing, in reference to Figure 2, we do see situations where consumers vary in their proclivity to enter the category (need arousal). Both the diffusion of innovation and hazard rate literatures address this problem (for example, see Roberts and Lattin 2000). Similarly, Fotheringham (1988) has used a fuzzy set approach to modeling consideration set membership in the information search stage to probabilistically describe whether a brand will be evoked. Next, it is in the modeling of beliefs (perceptions), preferences, and choice that probabilistic representations of consumer decision processes have really come into their own, with Hierarchical Bayes now used as a standard approach to modeling consumer differences (see, Rossi and Allenby 2003 for a review). Finally, as we move from the acquisition stages to the retention and value maximization ones, customer satisfaction models have used a variety of soft computing techniques to identify individual or segment-level threats and opportunities. Soft Computing Methods in Marketing: Phenomena and Management Problems Need Arousal Information search 23 Awareness Consideration Perceptions Evaluation Preference Purchase Post purchase Fig. 2 Lavidge and Steiner (1961)’s Hierarchy of Effects Model While soft computing has much to recommend it in each stage of the hierarchy of effects, where it has the most to offer is when these complexities are compounded. That is, while we can encounter large scale problems in each of these areas, it is the convolution of these large scale problems that particularly lends itself to the approach. Typical examples of such multi-level problems include the following: • • • Multidimensional consumer differences. We may have to segment on more than one basis (either within or between the levels of Figure 2). For example, within levels we may need to segment on the application to which the consumer is putting a service and her socio-economic profile. Between levels we may have to segment on the susceptibility of a consumer to an innovation at the need arousal level and the firm’s competitive capability at the purchase level. Multiple consumer purchases. The consumer may make multiple purchases within the category (suggesting a need to study share of wallet) or across categories (requiring estimation of cross-selling potential across multiple products). Interactions between consumers. Consumer networks may be critical, necessitating a study of order of magnitude of n2 with respect to customers, rather than just n (where n is the number of customers). 24 • J. Roberts Interactions between members of the market environment. Interactions between members of the channel, collaborators, competitors, and other groups (such as government regulators) may further compound the complexity of the problem. 3 Management Problems While multidimensional differences may exist at the level of the consumer or in the climate, they may not require complex models on the part of the manager to understand that variance. Before advocating a move to complex computing and modeling approaches, we must understand where just looking at the mean of distributions of heterogeneity is not going to lead to appropriate decisions, relative to a study of the entire distribution (or some middle approach such as looking at variances). Sometimes demand side factors alone may lead to a requirement to study the distribution of consumer tastes. The fallacy of averages in marketing is often illustrated by the fact that some people like iced tea, while others like hot tea. The fallacy of averages would suggest (incorrectly) that generally people like their tea lukewarm. In other situations, it is the context of managerial decision making in Figure 1 that makes complexity in the marketplace phenomena intractable to simplification and the use of means. The most obvious example of when modeling averages is problematic is when asymmetric loss functions exist: the benefit to the manager of upside error is not equal to the loss of downside. This will occur in a variety of situations. With lumpy investment decisions based on forecasts of consumer demand, over- and under-forecasts are likely to lead to very different consequences. Over-estimating demand is likely to lead to idle equipment and capital, while under-estimation will cause foregone contribution and possible customer dissatisfaction. Risk aversion on the part of the manager is another factor that will lead to asymmetric loss functions (Wehrung and Maccrimmon 1986). Finally, the presence of multiple decisions will lead to a requirement to study the whole distribution of customer outcomes, not just the mean. For example, in the ready to eat cereal and snacks market, Kellogg’s website lists 29 sub-brands1. Obviously, there are major interactions between these various sub-brands, and category optimization across them is an extremely complex problem. It is impossible to address without reference to the total distribution of beliefs, preferences and behaviors. Averages will not enable to answers to such portfolio management problems. 4 Summary Soft computing techniques have a number of advantages. Primarily, their ability to handle complex phenomena means that restrictive and potentially unrealistic assumptions do not need to be imposed on marketing problems. Balanced against 1 http://www2.kelloggs.com/brand/brand.aspx?brand=2 Soft Computing Methods in Marketing: Phenomena and Management Problems 25 this advantage is the loss of simplicity and parsimony, and this may incur associated costs of a loss of transparency and robustness. The mix of situations that favor soft computing techniques is increasing for a variety of reasons which may be understood by reference to Figure 1. Perhaps the primary drivers are trends in the market feedback tools available. Digital data capture means that large data sets are becoming available, enabling the modeling (and estimation) of consumer behavior in considerably finer detail than was previously possible. Computing power has obviously increased, and Moore’s law now enables calculations that would have been impossible, excessively onerous, or time intractable to be readily available. However, developments in both marketplace phenomena and managerial actions have also increased the potential application of soft computing approaches. Markets have become increasingly fragmented with the advent of highly targeted media and mass customization of products. For example, the U.K.’s largest retailer, Tesco, addresses over four million segments (Humby et al. 2008). In the top left box of Figure 1, managers have become increasingly sophisticated, with many firms employing sophisticated data mining techniques to address their customers. The emergence of specialist consulting and software firms (such as salesforce.com, dunhumby, and SAP) to support them has accelerated adoption in this area. Digitization has also increased the ability of the manager to experiment with a variety of strategies, leading to much richer mental models of the market, favoring the use of soft computing methods. Soft computing has the ability to lead us along paths that, as Keynes said, are more likely to be “vaguely right” rather than “precisely wrong” (e.g., Chick 1998). It is important that the migration path towards its use does not come at the cost of transparency or credibility. One way to ensure that this does not occur is to apply the techniques to environments that need the explanatory power they afford, and which influence management decisions for which the distribution of outcomes is critical, as well as the mean. References Chick, V.: On Knowing One’s Place: The Role of Formalism in Economics. The Economic Journal 108(451), 1859–1869 (1998) Day, G.S.: The Capabilities of Market-Driven Organizations. Journal of Marketing 58(4), 37–52 (1994) Stewart, F.A.: Consumer Store Choice and Choice Set Definition. Marketing Science 7(3) (Summer), 299–310 (1988) Humby, C., Hunt, T., Phillips, T.: Scoring Points: How Tesco Continues to Win Customer Loyalty. Kogan Page, London (2008) Jang, J.-S.R., Sun, C.-T., Mizutani, E.: Neuro-Fuzzy and Soft Computing-A Computational Approach to Learning and Machine Intelligence. Matlab Curriculum Series, Boston (1997) Lavidge, R.J., Steiner, G.A.: A Model for Predictive Measurements of Advertising Effectiveness. Journal of Marketing 25(6), 59–62 (1961) 26 J. Roberts Mitra, S., Pal, S.K., Mitra, P.: Data Mining in Soft Computing Framework: A Survey. IEEE Transactions On Neural Networks 13(1), 3–14 (2002) Roberts, J.H., Lattin, J.M.: Disaggregate Level Diffusion Models. In: Mahajan, V., Muller, E., Wind, Y. (eds.) New Product Diffusion Models, pp. 207–236. Kluwer Academic Publishers, Norwell (2000) Rossi, P.E., Greg, M.: Bayesian Statistics and Marketing. Marketing Science 22(3), 304–328 (Summer, 2003) Wehrung, D., Maccrimmon, K.R.: Taking Risks. The Free Press, New York (1986) User-Generated Content: The “Voice of the Customer” in the 21st Century* Eric T. Bradlow K.P. Chao Professor, Professor of Marketing, Statistics, and Education, Editor-in-Chief of Marketing Science, and Co-Director of the Wharton Interactive Media Initiative, University of Pennsylvania, Pennsylvania, USA 1 Introduction It doesn’t take an academic paper to point out the prominence that companies like Facebook, MySpace, YouTube, etc. have had on our popular culture today. Many see it as an efficient communication mechanism (Web 2.0 if you will) in comparison to email and static content postings which now, remarkably only 15 years later after the internet ‘launch’, some people see as “old school”. In some sense, Andy Warhol’s prediction of “15 minutes fame” for each and any one of us can now be “self-generated” through our own hard work and user-generated content. Thus, with all of this impact (societally) as backdrop, where does it leave many of us, as academics? The answer, and I hope this essay provides some impetus for that, is not on the sidelines. As a good sign, recently a joint call for funded research proposals between the Wharton Interactive Media Initiative (WIMI, www.whartoninteractive.com) and the Marketing Science Institute (MSI, www.msi.org) on the impact and modeling of user-generated content (UGC) generated an overwhelming response with over 50 submissions. Even better news was that these submissions were broad in their scope. As a non-random sampling of ideas generated, consider the following. • • * What is the impact of UGC on customer satisfaction and stock prices? Is there information contained in UGC that can help predict supra-normal returns? This can be considered, if you will, an update to the work of Fornell and colleagues (e.g. Anderson et. al, 1994), but now one based on UGC. How does the quantity and valence of UGC impact the diffusion (Bass, 1969) of new products? Note, that while the “scraping” of quantity information for the ‘amount’ of UGC may be somewhat simple, the valence of that information (‘quality’) is less so. While this may make the Financial support for this work was provided by the Wharton Interactive Media Initiative (www.whartoninteractive.com). J. Casillas & F.J. Martínez-López (Eds.): Marketing Intelligence Systems, STUDFUZZ 258, pp. 27–29. springerlink.com © Springer-Verlag Berlin Heidelberg 2010 28 E.T. Bradlow • timid shy away, this is one example of an opportunity where data mining and marketing scientists can partner together. Conjoint Analysis (Green and Rao, 1971) has long been a mainstay of marketing researchers as a method to understand consumer preferences for product features. But, how does one know that one has the right attributes in the first place – i.e. the classic “garbage in garbage out”? In a recent paper, Lee and Bradlow (2009) utilize feature extraction and clustering techniques to discover attributes that may be left off via standard focus group or managerial judgment methods. While these three examples are different in spirit, they all share a common theme, what can really be extracted from UGC that would aid decision-makers? In the next section, I discuss some thoughts, supportively encouraging and otherwise. 2 Marketing Scientists Should Care about UGC or Should They? Forecasting is big business. The ability to predict consumer’s actions in the future allows marketing interventions such as targeted pricing (Rossi et al, 1996), target promotions (Shaffer and Zhang, 2002), and the like. The promise that UGC can improve these marketing levers is certainly one reason that firms are investing heavily in data warehouses that can store this information and without this does UGC really have a “business future”? While it might seem tautological that UGC can help predict “who wants what” and “when”, it becomes less obvious when one conditions on past behavioral measures such as purchasing, visitation, etc… (Moe and Fader, 2004). In addition, what is the cost of keeping UGC at the individual-level? Thus, a new stream of research on data minimization methods, i.e. What is the least amount of information that needs to be kept for accurate forecasting? (Musalem et. al, 2006 and Fader et al, 2009) will soon, I believe, be at the forefront of managerial importance. Fear, created by the loss of not keeping something that may somehow, someday be useful, will be replaced by the guiding principles of parsimony and sufficiency (in the statistical sense and otherwise). Or, let us consider another example of UGC, viral spreading through social networks (Stephen and Lehmann, 2009). Does having content that users provide, knowing who their friends are, and how and to what extent they are sharing that information provide increased ability for targeted advertising? Does it provide the ability to predict “customer engagement” which can include pageviews, number of visits (Sismeiro and Bucklin, 2004), use of applications (now very popular on websites) and a firm’s ability to monetize it? These are open empirical questions which marketing scientists likely can not answer alone because of the widespread data collection that is necessary. We conclude next with a call for Data Mining and Marketing Science to converge. User-Generated Content: The “Voice of the Customer” in the 21st Century 29 3 Marketing Scientists and Data Mining Experts Need Each Other Now More Than Ever With all of the data that is abundant today, theory is now needed more than ever. Yes, I will say it again, theory is need now more than ever despite the belief of some that the massive amounts of data available today might make “brute empiricism” a solution to many problems. Without theory, all we are left with is exploration and sometimes massively unguided at that. Through the partnering of data mining/KDD experts in data collection and theory, and marketing scientists who can help link that data and theory to practice, UGC presents the next great horizon for “practical empiricism”. While the lowest hanging fruit might be including UGC covariates as predictors in models of behavior, hopefully our scientific efforts will move beyond that towards an understanding of its endogenous formation (the whys of people’s creation of it) and also an understanding of when it is truly insightful. References Anderson, E.W., Fornell, C., Lehmann, D.R.: Customer Satisfaction, Market Share, and Profitability: Findings from Sweden. Journal of Marketing 58(3), 53–66 (1994) Bass, F.M.: A New Product Growth Model for Consumer Durables. Management Science 15, 215–227 (1969) Fader, P.S., Zheng, Z., Padmanabhan, B.: Inferring Competitive Measures from Aggregate Data: Information Sharing Using Stochastic Models, Wharton School Working Paper (2009) Green, P.E., Rao, V.R.: Conjoint measurement for quantifying judgmental data. Journal of Marketing Research 8, 355–363 (1971) Griffin, A., Hauser, J.R.: The Voice of the Customer. Marketing Science 12(1), 1–27 (Winter 1993) Lee, T.Y., Bradlow, E.T.: Automatic Construction of Conjoint Attributes and Levels From Online Customer Reviews, Wharton School Working Paper (2009) Moe, W.W., Fader, P.S.: Dynamic Conversion Behavior at E-Commerce Sites. Management Science 50(3), 326–335 (2004) Musalem, A., Bradlow, E.T., Raju, J.: Bayesian Estimation of Random-Coefficients Choice Models using Aggregate Data. Journal of Applied Econometrics (2006) (to appear) Rossi, P.E., McCulloch, R.E., Allenby, G.M.: The Value of Purchase History Data in Target Marketing. Marketing Science 15, 321–340 (1996) Shaffer, G., Zhang, Z.J.: Competitive One-to-One Promotions. Management Science 48(9), 1143–1160 (2002) Sismeiro, C., Bucklin, R.E.: Modeling Purchase Behavior at an E-Commerce Web Site: A Task Completion Approach. Journal of Marketing Research, 306–323 (August 2004) Stephen, A.T., Lehmann, D.R.: Is Anyone Listening? Modeling the Impact of Word-ofMouth at the Individual Level, Columbia University Working Paper (2009) Fuzzy Networks Dawn Iacobucci E. Bronson Ingram Professor in Marketing, Owen Graduate School of Management, Vanderbilt University, Nashville, TN, USA Knowledge discovery and fuzzy logic have great potential for social network models. Networks are currently extraordinarily popular, and while it’s fun to work in an area that people find interesting, there is such a thing as a topic being too popular. Networks are too popular in the sense that they are not widely understood by users, hence they are thought to be the new, new thing, capable of answering all questions, from “Will my brand’s presence on Facebook help its equity?” to “ Will a network bring peace to the Middle East?” Fuzzy logic should help new users proceed from naïve enthusiasm to thoughtful application, because fuzzification embraces approximation; huge questions cannot be answered with simple, precise estimates, but a fuzzy approach can put the inquirer in the rough vicinity of an answer (Martínez-López and Casillas, 2008). This essay considers three popular uses of networks: word-of-mouth, brand communities, and recommendation agents, and the application of fuzziness in each realm. The first of these, word-of-mouth, has long been recognized as a powerful marketing force. Marketers routinely consider the diffusion of a new product or idea into and throughout the marketplace using models that posit the mechanism of customers informing each other. Those who adopt early are thought to influence the choices of those who adopt later. Hence, currently, the marketing question that seems to be the “holy grail” takes this form, “How can networks help me identify my influential customers?” This question is remarkably easy to answer via social network techniques. Actors in the network are assessed for their volume and strength of interconnections. Actors that are more interconnected with others are said to be “central” compared to more “peripheral” in the network. Depending on what the network ties reflect, centrality may manifest an actor’s importance, power, communication access, and the like. In a word-of-mouth network such as those sought in diffusion studies, these central players are the very essence of an influential opinion leader. There are several criteria to assess centrality, and as a result, indices abound (Knoke and Yang, 2007). For example, some measures reflect the sheer number of connections, or their weighted strengths or frequencies of connections. Other indices capture the extent to which actors are key in bridging multiple parts of the network map. Still other centrality measures reflect a sense of closeness among the network players, as in the number of steps between pairs of actors, or their “degrees of separation.” Nevertheless, the centrality indices share the property J. Casillas & F.J. Martínez-López (Eds.): Marketing Intelligence Systems, STUDFUZZ 258, pp. 31–34. © Springer-Verlag Berlin Heidelberg 2010 springerlink.com 32 D. Iacobucci that each captures the extent to which an actor has more connections to others, or stronger, or more frequently activated ties to others. These ties may be primarily inbound, and then the actor is said to be popular. The ties may be predominately outward bound, and then the actor is said to be expansive (e.g., extroverted). So where does fuzziness come in? Marketers understand that just because a customer engages in high activity, whether they claim many friends on a mobile phone plan, or are a frequent blogger, or actively recruit many friends on their Facebook page, it does not necessarily translate into their being an influential. But for all practical purposes, isn’t this status “close”? If someone posts to a blog, and some readers dismiss the posting as being uninformed, the marketer may be disappointed that this blogger isn’t as influential as first thought. Yet given their blogging volume and sheer statistical incidence, would it not likely be the case that their postings would impact some readers? Their blogging activity is presumably motivated by high customer involvement, thus may convey credibility, or at least passion. Thus, managing brand perceptions in the eyes of these frequent posters, frequent frienders, or frequent callers would be a social marketing activity whose result would be sufficiently close to the strategic aims of identifying and leveraging the influential customer. It is close enough. Brand communities are a contemporary marketing and social network phenomenon. Brand communities exist in real life and frequently online. People gather to share and learn and simply enjoy like-minded others. Some scholars claim they comprise a marketing or business strategy. I disagree. Marketing managers can try to launch such a community, and they can certainly insert marketing materials (brands, services, information) into the community in the hopes of effective persuasion. However, most authentic brand communities are grass-roots efforts, created by the love of a common element. For example, Harley riders got together long before some marketer coined the term, “brand community.” Marketing managers of brands that create such buzz and fondness can only hope to leverage the resulting community. Marketing managers of brands that create a collective yawn could persevere to eternity and not be successful in creating a community. When brand communities do exist, marketing phenomena such as diffusion can occur relatively quickly for two reasons. First, while the brand community can be rather large and its membership absolutely informal (e.g., no list of such actors exists), the community is still better defined and smaller to manage than the amorphous set of customers sought in the first application (of finding influentials among all customers). The marketer needs simply to be present at the auto / bike / beer / quilting event or website, and the community itself will take care of the information management, if it perceives value in the market offering. In addition, brand communities are largely democratic. In social network parlance, this egalitarian status shows itself distinctively in highly reciprocal or mutual ties. The ties create a clique of relatively highly interconnected actors comprising a subgroup within the network. Unlike the hierarchical relations between an early adopter exerting influence over a later adopter, customer elements in brand communities share mutual respect and communication. In such structures, those actors who extend ties in great volume tend to also receive them proportionally frequently. Fuzzy Networks 33 There is a lot to be learned from the patterns of social networks of brand communities—how do the Saturn and Harley communities compare? How do communities of brands whose customers are predominately women compare with those for men’s brands? How do Latin American constituted communities compare with networks for British brands and customers? And of course, is there a structural network distinction between communities of highly profitable brands and those that are less so?The very egalitarian nature of the brand community is related to the fuzziness principle for this social network phenomenon. Specifically, while it is true that members of a brand community are not created equal in terms of facility and likelihood of becoming a brand champion, it is not important. The marketing manager’s actions can be somewhat imprecise. If the marketer gets the brands and communications into the hands of the brand champion, diffusion will be rapid within the community. But even if the marketer misses, and the materials reach a proxy actor, doing so will eventually affect the same result, with the simple delay of the proxy communicating to the real community leaders. It is close enough. Finally, the third marketing phenomenon that can benefit from a fuzzy application of social networks is that of recommendation agents (Iacobucci, Arabie and Bodapati, 2000). Current data-based algorithms for suggesting new products to purchase or new hyperlinks to follow for related articles to read are based on clustering techniques. Social networks models can contribute to this pursuit in lending the concept and techniques of structural equivalence. Two actors are said to be structurally equivalent if they share the same pattern of ties to others. If we are willing to fuzz up this criterion, then two customers would be said to be stochastically equivalent if they share similar searches, purchases, or preference ratings. This third application is different from the first two in that they had been true social networks—the entities in a word-of-mouth network or in a brand community are predominately human, and the ties between these actors, social, be they communication links or ties of liking, respect, sharing, etc. The recommendation agency problem is contextualized in a mixed network of ties among entities that are human, electronic, tangible goods and brands and intangible services. The nonhuman actors may be said to be connected to the extent they are similar, bundled, complementary, etc. The human actors may be interconnected via the usual social ties, but may not be; the recommendation system in Amazon uses no friending patterns, but that in Netflix allows for others to make direct suggestions to people they know. For this phenomenon, like seeking influentials and seeding brand communities, fuzzy networks should suffice to yield good marketing results. Browsing book titles, music CDs, or movie DVDs in stores is easier than doing so online, yet a model-derived suggestion can put the customer in the ball-park of a new set of titles that may be of interest. Amazon’s statistics indicate that recommendations are not mindlessly embraced; e.g., the website offers indices such as, “After viewing this item, 45% of customers purchased it, whereas 23% of customers purchased this next item.” When one item is viewed, and another is suggested, the suggested item need not be embraced for the tool to be approximately useful. The suggested item puts the user down a new search path, restarting a nonrandom 34 D. Iacobucci walk. The user begins with a goal, which may be achieved immediately upon initial search, or it may be more optimally achieved upon corrected iteration based on inputs of recommendations resulting in successive approximations. Thus, we see that the system’s recommendation need not be “spot on.” Rather, the system only needs to be close enough. The study of network structures is a huge enterprise, and the application of networks to marketing and business phenomena is only in its infancy. These three examples were meant to be illustrative, drawing on popular and contemporary uses with which most readers will be familiar. Other examples at the nexus of fuzzy and networks will also benefit from the advantages of both. What was highlighted with the three exemplar fuzzy networks was the good news—that the application of networks does not need to be super precise for there to be great benefits realized. References Iacobucci, D., Arabie, P., Bodapati, A.: Recommendation Agents on the Internet. Journal of Interactive Marketing 14(3), 2–11 (2000) Knoke, D., Yang, S.: Social Network Analysis, 2nd edn. Sage, Thousand Oaks (2007) Martinez-Lopez, F.J., Casillas, J.: Marketing Intelligent Systems for Consumer Behavior Modeling by a Descriptive Induction Approach Based on Genetic Fuzzy Systems. Industrial Marketing Management Press (2008), doi:10.1016/j.indmarman.2008.02.003 KDD: Applying in Marketing Practice Using Point of Sale Information Adilson Borges1 and Barry J. Babin2 1 Reims Management School IRC Professor of Marketing Reims Cedex, France 2 Louisiana Tech University Reims Management School Max P. Watson, Jr. Professor of Business Chair, Department of Marketing and Analysis Louisiana Tech University, Ruston, LA, USA 1 Introduction The dramatic increase in computing power that has emerged over the past two to three decades has revolutionized decision making in most business domains. In particular, point of sale data has been recorded by retailers now since the time of scanner technology. However, the great volumes of data overwhelmed conventional computation routines until more recently. Although the basic principles of data mining can be found in automatic interaction detection routines dating back to the 1960s, the computational limitations of those days prevented a thorough analysis of all the possible combinations of variables. Today, KDD procedures are commonplace as data mining hardware and software provides power to search for patterns among practically any imaginable number of combinations. No longer do we talk about computer capacity in terms of megabytes, but more commonly, data storage is discussed in terms of terabytes (1000 gigabytes) or petabytes (1000 terabytes). Thus, although this may seem like an overwhelming amount of data and to be less theory driven than is appropriate for conventional multivariate data analysis procedures, it is clear that multivariate data analysis is applicable within soft computing and other data mining procedures (see Hair, Black, Babin and Anderson 2010). In particular, routines such as cluster analysis, multidimensional scaling, and factor analysis can be integrated into these routines to help establish patterns that can be validated and reduce the risk of identifying patterns based on randomly occurring generalizations. Retail management, like all marketing efforts, deals with decision making under conditions of uncertainty. This paper describes a KDD application from a retail setting. Managers constantly seek the best arrangement of products to maximize the value experience for consumers and maximize sales revenues for the retailer. Can KDD procedures assist in the store layout question? Here is a description of one attempt to do so. J. Casillas & F.J. Martínez-López (Eds.): Marketing Intelligence Systems, STUDFUZZ 258, pp. 35 – 41. © Springer-Verlag Berlin Heidelberg 2010 springerlink.com 36 A. Borges and B.J. Babin This paper proposes a new grocery store layout based on the association among categories. We use the buying association measure to create a category correlation matrix and we apply the multidimensional scale technique to display the set of products in the store space. We will imply that the buying association, measured through the market basket analysis, is the best way to find product organization that are best suited to one stop shopping. 2 The Store Layout Increasing space productivity represents a powerful truism in retailing: customers buy more when products are merchandised better. By careful planning of the store layout, retailers can encourage customers to flow through more shopping areas, and see a wider variety of merchandise (Levy and Weitz, 1998). There are at least two layout approaches: the traditional grid layout and the consumption universe layout. The traditional approach consists in repeating the industrial logic implementation, which means putting products that share some functional characteristics or origins in the same area. So we will find the bakery area (with bread, cakes, biscuits, etc), the vegetable area (with carrots, beans, etc), and so on. This traditional approach has been improved by the use of cross-elasticities, which should measure use association. Retailers have changed some categories and put more complementary in use items together. If a consumer wants take photos at a family party, s/he needs at least the camera and the film. In these cases, both products are complementary, because consumers need both at same time to achieve a specific goal (Walters, 1991). The nature of the relationship among products could be twofold: the use association (UA) or the buying association (BA). UA is the relationship among two or more products that meet specific consumer need by their functional characteristics. We can classify the relationship among different categories by their uses: the products can be substitutes, independent and complementary (Henderson and Quandt, 1958 ; Walter, 1991). The BA is the relationship established by consumers through their transaction acts and it will be verified in the market basket. While UA is not a necessary condition for BA, because UA depends much more on the products functional characteristics, BA depends on buying and re-buying cycles as well as on store marketing efforts. Despite improvements, the store remains organized in “product categories” as defined by the manufacturers or category buyers. This approach is company oriented and it fails to respond to the needs of the time pressured consumer. Some retailers are trying to move from this organization to something new, and are trying to become ¨consumer oriented¨ in their layout approach. Tesco has rethought their store layout with ¨plan-o-grams¨ to try to reflect local consumers needs (Shahidi, 2002). Other French retailers have used consumption universe layouts to make it easier for consumers to find their product in a more hedonic environment. This approach allows supermarkets to cluster products around meaningful purchase opportunities related to use association. Instead of finding coffee in the beverage section, cheese in fresh cheese, ham in the meat section, and cornflakes in KDD: Applying in Marketing Practice Using Point of Sale Information 37 the cereal section, we could find all those products in the breakfast consumption universe. Other universes, such as the baby universe or tableware universe, propose the same scheme to cluster different product categories. It is too soon to foresee the financial results of such applications, but it shows, however, the retailer’s desire to improve in store product display. These new layout applications do not take the one stop shop phenomenon into account. In fact, this approach is based on the principle that conjoint use of products will unconditionally produce conjoint buying. The main problem with this rationale is that use association alone cannot be used to explain the associations carried out in the buying process (the market basket), because it fails to take buying time cycles into account. For example, bread and butter should be classified as occasional complements, and then they should be found in the same market basket (Walters, 1991). However, this could be not true, since the products have different buying and re-buying cycles. In that case, buying association may be weak, because bread is usually bought on a daily basis, and butter once every week or two. On the other hand, ‘independent products’ don’t have any use relationship, so they should not have any stable buying association. Meanwhile, Betancourt and Gautschi (1990) show that some products could be bought at the same time as a result of the store merchandising structure, store assortment, the marketing efforts and consumption cycles. So, the fact that two products are complementary is not a guarantee that those products will be present in the same market basket. In addition, some researchers have found that independent products have the same correlation intensity as complementary ones in the market baskets (Borges et alli, 2001). So, the store layout construction has to incorporate the market basket analysis to improve the one stop shopping experience. This allows retailers to cluster products around the consumer buying habits, and then to create a very strong appeal for today’s busy consumers. 3 The Buying Association: A Way to Measure the Relationship among Products The relationship between categories has always been articulated through their use, but this is not enough to explain conjoint presence in the market basket. These two kinds of relationships were clear for Balderston (1956), who presented it as (1) use complementary, if products are used together, and (2) buying complementary, if products are bought together. BA can be computed from supermarket tickets, and indicates real consumer behavior (it is not based on consumers’ declaration or intention). Loyalty cards and store scanners have produced a huge amount of data that is stored in data warehouses and analyzed by data mining techniques. Data Mining is regarded as the analysis step in the Knowledge Discovery in Databases (KDD) process, which is a "non-trivial process of extracting patterns from data that are useful, novel and comprehensive". In data mining, BA is considered as an association rule. This 38 A. Borges and B.J. Babin association rule is composed of an antecedent and consequence set : A ⇒ B, where A is an antecedent and B a consequent; or A,B ⇒ C, where there are two antecedents and one consequence (Fayyad et alli, 1996). The BA is calculated by the following formula: δ AB = f ( AB) , f ( A) (1) where f(AB) represents the conjoint frequency of both products A and B and f(A) represents the product A frequency in the database. This equation is similar to the conditional probability that could be written as (A∩B)/A, given that A intersection B represents the market baskets where both products, A and B, are present at same time. The buying association represents the percentages of consumers that buy product A and who also buy product B. It shows the relationship strength between products, considering only the relationships carried out on buying behavior. This can be represented as a percentage: a BA of 35% between coffee and laundry is interpreted as 35% of consumers have bought coffee also bought laundry in the same shopping trip. In the same way that cross-elasticity is not symmetric, BA is also not symmetric. The BAFC can be different from BACF (this relationship depends mainly on the category penetration rates over the total sales). Mathematically: ∀ F>C, so (F∩C)/F < (F∩C)/C. So, if A frequency is different from B frequency, then the relationship among those products will always be asymmetric. For example, “F” represents the film and “C” the camera. Suppose the condition F>C is confirmed, then the film has a larger penetration in the market baskets than camera. If this condition is satisfied, then BAFC