Scalable Progressive Analytics on Big Data in the Cloud Badrish Chandramouli1Jonathan Goldstein1Abdul Quamar2∗1Microsoft Research,Redmond2University of Maryland,College Park{badrishc,jongold}@,abdul@ABSTRACTAnalytics over the increasing quantity of data stored in the Cloud has become very expensive,particularly due to the pay-as-you-go Cloud computation model.Data scientists typically manually ex-tract samples of increasing data size(progressive samples)using domain-specific sampling strategies for exploratory querying.This provides them with user-control,repeatable semantics,and result provenance.However,such solutions result in tedious workflows that preclude the reuse of work across samples.On the other hand, existing approximate query processing systems report early results, but do not offer the above benefits for complex ad-hoc queries.We propose a new progressive analytics system based on a progress model called Prism that(1)allows users to communicate progres-sive samples to the system;(2)allows efficient and deterministic query processing over samples;and(3)provides repeatable seman-tics and provenance to data scientists.We show that one can re-alize this model for atemporal relational queries using an unmod-ified temporal streaming engine,by re-interpreting temporal event fields to denote progress.Based on Prism,we build Now!,a pro-gressive data-parallel computation framework for Windows Azure, where progress is understood as afirst-class citizen in the frame-work.Now!works with“progress-aware reducers”-in particular,it works with streaming engines to support progressive SQL over big data.Extensive experiments on Windows Azure with real and syn-thetic workloads validate the scalability and benefits of Now!and its optimizations,over current solutions for progressive analytics.1.INTRODUCTIONWith increasing volumes of data stored and processed in the Cloud,analytics over such data is becoming very expensive.The pay-as-you-go paradigm of the Cloud causes computation costs to increase linearly with query execution time,making it possible for a data scientist to easily spend large amounts of money analyzing data.The problem is exacerbated by the exploratory nature of ana-lytics,where queries are iteratively discovered and refined,includ-ing the submission of many off-target and erroneous queries(e.g., bad parameters).In traditional systems,queries must execute to ∗Work performed during internship at Microsoft Research. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on thefirst page.To copy otherwise,to republish,to post on servers or to redistribute to lists,requires prior specific permission and/or a fee.Articles from this volume were invited to present their results at The39th International Conference on Very Large Data Bases, August26th-30th2013,Riva del Garda,Trento,Italy.Proceedings of the VLDB Endowment,Vol.6,No.14Copyright2013VLDB Endowment2150-8097/13/14...$pletion before such problems are diagnosed,often after hours of expensive compute time are exhausted.Data scientists therefore typically choose to perform their ad-hoc querying on extracted samples of data.This approach gives them the control to carefully choose from a huge variety[11,28,27]of sampling strategies in a domain-specific manner.For a given sam-ple,it provides precise(e.g.,relational)query semantics,repeatable execution using a query processor and optimizer,result provenance in terms of what data contributed to an observed result,and query composability.Further,since choosing afixed sample size a pri-ori for all queries is impractical,data scientists usually create and operate over multiple progressive samples of increasing size[28].1.1ChallengesIn an attempt to help data scientists,the database community has proposed approximate query processing(AQP)systems such as CONTROL[20]and DBO[23]that perform progressive analytics. We define progressive analytics as the generation of early results to analytical queries based on partial data,and the progressive refine-ment of these results as more data is received.Progressive analytics allows users to get early results using significantly fewer resources, and potentially end(and possibly refine)computations early once sufficient accuracy or query incorrectness is observed.The general focus of AQP systems has,however,been on auto-matically providing confidence intervals for results,and selecting processing orders to reduce bias[21,9,15,17,31].The premise of AQP systems is that users are not involved in specifying the seman-tics of early results;rather,the system takes up the responsibility of defining and providing accurate early results.To be useful,the system needs to automatically select effective sampling strategies for a particular combination of query and data.This can work for narrow classes of workloads,but does not generalize to complex ad-hoc queries.A classic example is the infeasibility of sampling for join trees[10].In these cases,a lack of user involvement with “fast and loose”progress has shortcomings;hence,data scientists tend to prefer the more laborious but controlled approach presented earlier.We illustrate this using a running example.Example1(CTR)Consider an advertising platform where an an-alyst wishes to compute the click-through-rate(CTR)for each ad. We require two sub-queries(Q c and Q i)to compute(per ad)the number of clicks and impressions,respectively.Each query may be non-trivial;for example,Q c needs to process clicks on a per-user basis to consider only legitimate(non-automated)clicks from a webpage whitelist.Further,Q i may need to process a different set of logged data.Thefinal query Q ctr joins(for each ad)the results of Q c and Q i,and computes their ratio as the CTR.Figure1shows a toy input sorted by user,and thefinal results for Q c,Q i,and Q ctr.User Ad...u 0a 0...u 1a 0...u 2a 0...(a)User Ad ...u 0a 0...u 0a 0...u 1a 0...u 2a 0...u 2a 0...(b)Ad Clicks a 03Ad Imprs a 05(c)Ad CTR a 00.6(d)Figure 1:(a)Click data;(b)Impression data;(c)Final result of Q c and Q i ;(d)Final result of Q ctr .Next,Figure 3(a)and (b)show progressive results for the same queries Q c and Q i .Without user involvement in defining progres-sive samples,the exact sequence of progressive counts can be non-deterministic across runs,although the final counts are precise.Further,depending on the relative speed and sequence of results for Q c and Q i ,Q ctr may compose arbitrary progressive results,re-sulting in significant variations in progressive CTR results.Fig-ures 3(c)and (d)show two possible results for Q ctr .For example,a CTR of 2.0would result from combining the first tuple from Q c and Q i .Some results that are not even meaningful (e.g.,CTR >1.0)are possible.Although both results eventually get to the same final CTR,there is no mechanism to ensure that the inputs being corre-lated to compute progressive CTRs are deterministic and compara-ble (e.g.,computed using the same sample of users).The above example illustrates several challenges:1)User-Control :Data scientists usually have domain expertise that they leverage to select from a range of sampling strategies [11,28,27]based on their specific needs and context.In Example 1,we may progressively sample both datasets identically in user-order for meaningful progress,avoiding the join sampling problem [10].Users may also need more flexibility;for instance,with a star-schema dataset,they may wish to fully process the small dimension table before sampling the fact table,for better progressive results.2)Semantics :Relational algebra provides precise semantics for SQL queries.Given a set of input tables,the correct output is de-fined by the input and query alone,and is independent of dynamic properties such as the order of processing tuples.However,for complex queries,existing AQP systems use operational semantics ,where early results are on a best-e ffort basis.Thus,it is unclear what a particular early result means to the user.3)Repeatability &Optimization :Two runs of a query in AQP may provide a di fferent sequence of early results,although they have to both converge to the same final answer.Thus,without limiting the class of queries which are progressively executed,it is hard to understand what early answers mean,or even recognize anomalous early answers.Even worse,changing the physical operators in the plan (e.g.,changing operators within the ripple join family [16])can significantly change what early results are seen!4)Provenance :Users cannot easily establish the provenance of early results,e.g.,link an early result (CTR =3.0)to particular con-tributing tuples,which is useful to debug and reason about results.5)Query Composition :The problem of using operational seman-tics is exacerbated when one starts to compose queries.Example 1shows that one may get widely varying results (e.g.,spurious CTR values)that are hard to reason about.6)Scale-Out :Performing progressive analytics at scale exacer-bates the above challenges.The CTR query from Example 1is Figure 2:CTR;MR jobs.expressed as two map-reduce (MR )jobs that partition data by UserId,feeding a third job that partitions data by a di fferent key (AdId);see Figure 2.In a complex distributed multi-stageworkflow,accurate deterministicprogressive results can be veryAd Clicks a 02a 03(a)Ad Imprs a 01a 04a 05(b)Ad CTR a 02.0a 00.5a 00.6(c)Ad CTR a 03.0a 00.75a 00.6(d)Figure 3:(a)Progressive Q c output;(b)Progressive Q i output;(c)&(d)Two possible progressive Q ctr results.useful.Map-reduce-online (MRO )[12]adds a limited form of pipelining to MR,but MRO reports a heuristic progress metric (av-erage fraction of data processed across mappers)that does not elim-inate the problems discussed above (§6covers related work).To summarize,data scientists prefer user-controlled progressive sampling because it helps avoid the above issues,but the lack of system support results in a tedious and error-prone workflow that precludes the reuse of work across progressive samples.We need a system that (1)allows users to communicate progressive samples to the system;(2)allows e fficient and deterministic query processing over progressive samples,without the system itself trying to reason about specific sampling strategies or confidence estimation;and yet (3)continues to o ffer the desirable features outlined above.1.2Contributions1)Prism :A New Progress Model &ImplementationWe propose (§2)a new progress model called Prism (Progressive sampling model ).The key idea is for users to encode their cho-sen progressive sampling strategy into the data by augmenting tu-ples with explicit progress intervals (PIs ).PIs denote logical points where tuples enter and exit the computation,and explicitly assign tuples to progressive samples.PIs o ffer remarkable flexibility for encoding sampling strategies and ordering for early results,includ-ing arbitrarily overlapping sample sequences and special cases such as the star-schema join mentioned earlier (§2.5has more details).PIs propagate through Prism bined with progres-sive operator semantics,PIs provide closed-world determinism:the exact sequence of early results is a deterministic function of aug-mented inputs and the logical query alone.They are independent of physical plans,which enables side-e ffect-free query optimization.Provenance is explicit;result tuples have PIs that denote the ex-act set of contributing inputs.Prism also allows meaningful query composition,as operators respect PIs.If desired,users can encode confidence interval computations as part of their queries.The introduction of a new progress model into an existing re-lational engine appears challenging.However,interestingly,we show (§2.4)that a progressive in-memory relational engine based on Prism can be realized immediately using an unmodified tem-poral streaming engine ,by carefully reusing its temporal fields to denote progress.Tuples from successive progressive samples get incrementally processed when possible,giving a significant perfor-mance benefit.Note here that the temporal engine is unaware that it is processing atemporal relational queries;we simply re-interpret its temporal fields to denote progress points.While it may appear that in-memory queries can be memory intensive since the final answer is computed over the entire dataset,Prism allows us to ex-ploit sort orders and foreign key dependencies in the input data and queries to reduce memory usage significantly (§2.6).Prism generalizes AQP Our progress semantics are compatible with queries for which prior AQP techniques with statistical guar-antees apply,and thus don’t require user involvement.These tech-niques simply correspond to di fferent PI assignment policies for input data.For instance,variants of ripple join [16]are di fferent PI assignments for a temporal symmetric-hash-join,with confidence intervals computed as part of the query.Thus,Prism is orthogo-nal to and can leverage this rich area of prior work,while addingPI User Ad [0,∞)u0a0 [1,∞)u1a0 [2,∞)u2a0(a)PI User Ad[0,∞)u0a0[0,∞)u0a0[1,∞)u1a0[2,∞)u2a0[2,∞)u2a0(b)PI Ad Clicks[0,1)a01[1,2)a02[2,∞)a03PI Ad Imprs[0,1)a02[1,2)a03[2,∞)a05(c)PI Ad CTR[0,1)a00.5[1,2)a00.66[2,∞)a00.6(d)Figure4:(a,b)Input data with progress intervals;(c)Progres-sive results of Q c and Q i;(d)Progressive output of Q ctr.the benefit of repeatable and deterministic semantics.In summary, Prism gives progressive results the same form of determinism and user control that relational algebra providesfinal results.2)Applying Prism in a Scaled-Out Cloud SettingThe Prism model is particularly suitable for progressive analytics on big data in the Cloud,since queries in this setting are complex, and memory-and CPU-intensive.Traditional scalable distributed frameworks such as MR are not pipelined,making them unsuitable for progressive analytics.MRO adds pipelining,but does not offer the semantic underpinnings of progress necessary to achieve the desirable features outlined earlier.We address this problem by designing andbuilding a new frame-work for progressive analytics called Now!(§3).Now!runs on Windows Azure;it understands and propagates progress(based on the Prism model)as afirst-class citizen inside the framework. Now!generalizes the popular data-parallel MR model and supports progress-aware reducers that understand explicit progress in the data.In particular,Now!can work with a temporal engine(we use StreamInsight[3])as a progress-aware reducer to enable scaled-out progressive relational(SQL)query support in the Cloud.Now!is a novel contribution in its own right,with several important features:•Fully pipelined progressive computation and data movement across multiple stages with different partitioning keys,in order to avoid the high cost of sending intermediate results to Cloud storage.•Elimination of sorting in the framework using progress-ordered data movement,partitioned computation pushed inside progress-aware reducers,and support for the traditional reducer API.•Progress-based merge of multiple map outputs at a reducer node.•Concurrent scheduling of multi-stage map and reduce jobs with a new scheduling policy andflow control scheme.We also extend Now!(§4)with a high performance mode that eliminates disk writes,and discuss high availability(by leveraging progress semantics in a new way)and straggler management.We perform a detailed evaluation(§5)of Now!on Windows Azure(with StreamInsight)over real and benchmark datasets up to 100GB,with up to75large-sized Azure compute instances.Exper-iments show that we can scale effectively and produce meaningful early results,making Now!suitable in a pay-as-you-go environ-ment.Now!provides a substantial reduction in processing time, memory and CPU usage as compared to current schemes;perfor-mance is significantly enhanced by exploiting sort orders and using our memory-only processing mode.Paper Outline§2provides the details of our proposed model for progressive computation Prism.We present Now!in detail in §3;and discuss several extensions in§4.The detailed evaluation of Now!is covered in§5,and related work is discussed in§6. 2.Prism SEMANTICS&CONSTRUCTION At a high level,our progress model(called Prism)defines a log-ical linear progress domain that represents the progress of a query. Sampling strategies desired by data scientists are encoded into the data before query processing,using augmented tuples with progressImpressionInput DataProgress Domain1 2ClickInput DataFigure5:Input and output progress intervals,query semantics. intervals that precisely define how data progressively contributes to result ers express their data analytics as relational queries that consist of a DAG of progressive operators.An exten-sion of traditional database operators,progressive operators under-stand and propagate progress intervals based on precisely defined operator semantics.The result of query processing is a sequence of augmented tuples whose progress intervals denote early results and their associated regions of validity in the progress domain.Each of these steps is elaborated in the following subsections.2.1Logical Progress and Progress Intervals Prism defines a logical linear progress domain P as the range of non-negative integers[0,∞).Progress made by a query at any given point during computation is explicitly indicated by a non-decreasing progress point p∈P.Progress point∞indicates the point of query completion.Next,we associate a progress interval(PI)from the progress do-main to every tuple in the input data.More formally,each tuple T is augmented with two new attributes,a progress-start P+and a progress-end P-,that jointly denote a PI[P+,P-).P+indicates the progress point at which a tuple T starts participating in the compu-tation,and P-(if not∞)denotes the progress point at which tuple T stops contributing to the computation.PIs enable users to spec-ify domain-specific progressive sampling strategies.PI assignment can be controlled by data scientists to ensure quicker and more meaningful early results,either directly or using a layer between the system and the user.Figures4(a)and(b)show PIs for our run-ning example inputs;they are also depicted in Figure5(top).We provide several concrete examples of PI assignment in Section2.5.2.2Progressive Operators and Queries Progressive Operators Every relational operator O has a progres-sive counterpart,which computes augmented output tuples from augmented input tuples.Logically,the output at progress point p is the operation O applied to input tuples whose PIs are stabbed by p. Figures4(c)and5(bottom)show the results of Q c and Q i,which behave as Count operators.We see that Q c produces a progressive count of1at progress point0,which it revises to2and3at progress points1and2.As a result,the PIs for these tuples are[0,1),[1,2) and[2,∞)respectively.The P-for an output tuple may not always be known at the same time as when the operator determine its P+.Thus,an operator may output a tuple having an eventual PI of[P+,P-)in two separate pieces:(1)at progress point P+,it generates a start-edge tuple T1 with a PI[P+,∞)indicating that the tuple participates in the result forever;(2)at the later progress point P-,it generates an end-edge tuple T2with the actual PI[P+,P-).We use the term progress-sync to denote the progress point associated with a tuple(or its subse-quent update).The start-edge tuple T1has a progress-sync of P+, whereas the end-edge tuple T2has a progress-sync of P-.Every operator both processes and generates augmented tuples in non-decreasing progress-sync order.The eventual P-values for early results that get refined later are less than∞,to indicate that the result is notfinal.For example,consider an Average operator that reports a value a0from progress point0to10,and revises it to a1from progress point10onwards.Tuple a0has an eventual PI of[0,10).This is reported as a start-edge[0,∞)at progress point 0.At progress point10,the operator reports an end-edge[0,10)for the old average a0,followed immediately by a start-edge[10,∞)for the revised average a1.Similarly,a progressive Join operator with one tuple on each input with PIs[10,20)and[15,25)–if the join condition is satisfied–produces a result tuple with PI[15,20),the intersection of the two input PIs.Note here that the output tuple’s PI ends at20because its left input is no longer valid at that point. Progressive Queries Based on the above semantics,operators can be composed meaningfully to produce progressive queries.We define Prism output for a relational query Q as:Definition1(Prism Output)Associated with each input tuple is a progress interval(PI).At every unique progress point p across all PI endpoints in the input data,there exists a set O p of output results with PIs stabbed by p.O p is defined to be exactly the result of the query Q evaluated over input tuples with PIs stabbed by p.2.3Summary of Benefits of the Prism Model The results of Q ctr for our running example are shown in Fig-ures4(d)and5(bottom);every CTR is meaningful as it is com-puted on some prefix of users(for our chosen progress assignment), and CTR provenance is provided by PIs.Thefinal CTR of0.6is the only tuple active at progress point∞,as expected.It is easy to see that the output of a progressive query is a de-terministic function of the(augmented)input data and the logical query alone.Further,these progressive results arefixed for a given input and logical query,and are therefore repeatable.Prism en-ables data scientists to use their domain knowledge to control pro-gressive samples;Section2.5provides several concrete examples. Early results in Prism carry the added benefit of provenance that helps debug and reason about early results:the set of output tuples with PIs stabbed by progress point p denote the progressive result of the query at p.The provenance of these output tuples is simply all tuples along their input paths whose PIs are stabbed by p.One can view Prism as a generalization of relational algebra with progressive sampling as afirst-class concept.Relational algebra prescribes thefinal answer to a relational query but does not cover how we get there using partial results.The Prism algebra explicitly specifies,for any query,not only thefinal answer,but every inter-mediate(progressive)result and its position in the progress domain.2.4Implementing PrismOne can modify a database engine to add PI support to all oper-ators in the engine.However,we can realize Prism without incur-ring this effort.The idea is to leverage a stream processing engine (SPE)as the progressive query processor.In particular,the seman-tics underlying a temporal SPE such as NILE[19],STREAM[4], or StreamInsight[3](based on temporal databases[22])can be leveraged to denote progress,with the added benefit of incremen-tal processing across samples when possible.With StreamInsight’s temporal model,for example,the event validity time interval[6] [V s,V e)directly denotes the PI[P+,P-).T1is an insertion and T2 is a retraction(or revision[33]).Likewise,T1and T2correspond to Istreams and Dstreams in STREAM,and positive and negative tuples in NILE.We feed the input tuples converted into events to a continuous query corresponding to the original atemporal SQL query.The unmodified SPE operates on these tuples as though they were temporal events,and produces output events with timestamp fields that we re-interpret as tuples with PIs.Note that with this construction,the SPE is unaware that it is being used as a progressive SQL processor.It processes and pro-duces events whose temporalfields are re-interpreted to denote progress of an atemporal(relational)query.For instance,the tem-poral symmetric-hash-join in an SPE effectively computes a se-quence of joins over a sequence of progressive samples very ef-ficiently.The resulting query processor transparently handles all of SQL,including user-defined functions,with all the desirable fea-tures of our new progress model.2.5PI AssignmentAny progressive sampling strategy at the inputs corresponds to a PI assignment;several are discussed next.Inclusive&Non-inclusive Samples With inclusive samples(as used,for example,in EARL[25]),each sample is a superset of the previous one.To specify these,input tuples are assigned a P-of∞, and non-decreasing P+values based on when tuples become a part of the sample,as shown in Figure5(top).In case of non-inclusive samples,tuples have afinite P-to denote that they no longer par-ticipate in computation beyond P-,and can even reappear with a greater P+for a later sample(our technical report[7]includes a concrete example of expressing non-inclusive sampling using PIs). Reporting Granularity Progress reporting granularity can be controlled by individual queries,by adjusting the way P+moves forward.Data is often materialized in a statistically relevant order, and we may wish to include k additional tuples in each successive sample.We use a streaming AlterLifetime[8]operator that sets P+ for the n th tuple to n/k and P-to∞.This increases P+by1after every k tuples,resulting in the engine producing a new progressive result every k tuples.We refer to the set of tuples with the same P+ as a progress-batch.Data scientists often start with small progress-batches to get quick estimates,and then increase batch sizes(e.g., exponentially)as they get diminishing returns with more data. Joins&Star Schemas In case of queries involving an equi-join, we may apply an identical sampling strategy(e.g.,pseudo-random) over the join key in both inputs as this increases the likelihood of getting useful early results.With a star-schema,we may set all tu-ples in the small dimension table to have a PI of[0,∞),while pro-gressively sampling from the fact table as[0,∞),[1,∞),....This causes a Join operator to“preload”the dimension table before pro-gressively sampling the fact table for meaningful early results. Stratified Sampling Stratified sampling groups data on a cer-tain key and applies a sampling strategy(e.g.,uniform)within each group to ensure that rare subgroups are sufficiently represented. BlinkDB[2]pre-computes stratified samples of different sizes and responds to queries within a given error and response time by choos-ing the correct sample to compute the query on.Stratified sampling is easy to implement with Prism:we perform a GroupApply opera-tion[8]by the key,with an AlterLifetime inside the GroupApply to create progress-batches as before.The temporal Union that merges groups respects timestamp ordering,resulting in afinal dataset with PIs that exactly represent stratified sampling.Stratified samples of increasing size can be constructed similarly.Other Examples For online aggregation,we may assign non-decreasing P+values over a pre-defined random order of tuples for quick result convergence.Active learning[11]changes the sam-pling strategy based on outcomes from prior samples.Prior propos-als for ordering data for quick convergence[17,16,30,9]simply correspond to different PI assignment schemes in Prism.Blob Blob BlobMap StageDisk Disk DiskReduce StageMergeMerge Merge Map StageDisk Reduce StageMergeMerge Merge Blob Blob Blob ShuffleShuffleDisk DiskSortBlobBlob BlobMap Stage(Progress-aware batching)Progressive Reducer (Gen API)Progress Aware Merge Map Stage(Progress-aware batching)Progressive Reducer (Gen API)Progress Aware Merge BlobBlob BlobProgressive datashuffleNo SortBlob Blob BlobIn Memory data transferProgressive datashuffleNo Sort(a) Vanilla MR(b) Now!annotated InputProgressive Output Figure 6:System architecture (MR vs.).2.6Performance OptimizationsQuery processing using an in-memory streaming engine can be expensive since the final answer is over the entire dataset.Prism enables crucial performance optimizations that can improve perfor-mance significantly in practical situations.Consider computation Q c ,which is partitionable by UserId.We can exploit the compile-time property that progress-sync ordering is the same as (or cor-related to)the partitioning key,to reduce memory usage and con-sequently throughput.The key intuition is that although every tu-ple with PI [P +,∞)logically has a P -of ∞,it does not contribute to any progress point beyond P +.Thus,we can temporarily set P -to P ++1before feeding the tuples to the SPE.This e ffectively causes the SPE to not have to retain information related to progress point P +in memory once computation for P +is done.The result tuples have their P -set back to ∞to retain the original query se-mantics (these query modifications are introduced using compile-time query rewrites).A similar optimization applies to equi-joins;see [7]for details.We will see in Section 5that this optimization can result in orders-of-magnitude performance benefits.We next discuss our new big data framework called Now!,that implements Prism for MR-style computation at scale in a distributed setting (applying Prism to other models such as graphs is an inter-esting area of future work).3.Now!ARCHITECTURE AND DESIGN3.1OverviewAt a high level,Now!’s architecture is based on the Map-Reduce (MR)[14]computation paradigm.Figure 6shows the overall de-sign of Now!(right)as compared to vanilla MR (left),for a query with two stages and di fferent partitioning keys.Blobs in the figure indicate the format of input and output data on Windows Azure’s distributed Cloud storage,and can be replaced by any distributed persistent storage such as HDFS.The key points are as follows:1)Progress-aware data flow:Now!implements the Prism progress model and provides support for data flow (§3.2)in strict progress-sync order.The main components of progress-aware data flow are:•Batching Now!reads input data annotated with PIs (progres-sive samples)and creates batches (§3.2.1)of tuples with thein Now!is fully pipelined progress-sync order.map output by key,fol-by key at reducers.This bottleneck [26].In is partitioned and shuf-sorting (§3.2.2),thus performance.merge at reducers is progressive query results.received from di fferent a single progress-batch,processed in strict progress-paths.in Now!uses TCP connec-the input data is read retain this invariant,we for progress-batches.the notion of a progress-and produces augmented adheres to the Prism generates progress-batches in progress-sync order;these are fed directly to reducers that pro-duce early results in progress-sync order.While one could write custom reducers,we use an unmodified SPE (§2.4)as a progress-aware reducer for progressive relational queries.3)Multi-stage support:Now!supports concurrent scheduling of all jobs in a multi-stage query and co-location of mappers of de-pendent jobs with the reducers of feeding jobs on the same slave machine (Section 3.3).Data transfer between jobs is in-memory providing significant savings in a Cloud deployment where blob access is expensive.4)Flow control:Now!provides end-to-end flow control to avoid bu ffer overflows at intermediate stages such as mapper output,re-ducer input and reducer output for multi-stage MR.The flow con-trol mechanism ensures data flows at a speed that can be sustained by downstream consumers.We use a blocking concurrent queue (BCQ ),a lock-free data structure which supports concurrent en-queue and dequeue operations,for implementing an end-to-end flow control mechanism for Now!(our technical report [7]has more details on flow control in Now!).5)In-memory data processing:By default,Now!materializes map output on disk to provide better data availability during failure re-covery.For better interactivity,we also support a high-performance in-memory mode (see Section 4).3.2Progress-aware data flow &computationData flow in Now!is at the granularity of progress-batches and governed by PIs.This section describes the generation and flow of these progress-batches in the framework.3.2.1Progress-aware batchingThe input data is partitioned into a number of input splits (one for each mapper),data tuples in each of which are assigned progress intervals in progress-sync order.The mapper reads its input split as progress annotated tuples (progressive samples),and invokes the user’s map function.The resulting augmented key-value pairs are partitioned by key to produce a sequence of progress-batches for each partition (downstream reducer).A progress-batch consists of all tuples with the same progress-sync value (within the specific partition)and has a unique ID.Each progress-batch sequence is in strictly increasing progress-sync order.The input text reader appends an end-of-file (eof )marker to the mapper’s input when it。
训练集（trainset）验证集（validationset）测试集（testset）。
训练集(train set) 验证集(validation set) 集(test set)。
⼀般需要将样本分成独⽴的三部分训练集(train set)，验证集(validation set)和测试集(test set)。
其中训练集⽤来估计模型，验证集⽤来确定⽹络结构或者控制模型复杂程度的参数，⽽测试集则检验最终选择最优的模型的性能如何。
⼀个典型的划分是训练集占总样本的50％，⽽其它各占25％，三部分都是从样本中随机抽取。
样本少的时候，上⾯的划分就不合适了。
常⽤的是留少部分做测试集。
然后对其余N个样本采⽤K折交叉验证法。
就是将样本打乱，然后均匀分成K份，轮流选择其中K－1份训练，剩余的⼀份做验证，计算预测误差平⽅和，最后把K次的预测误差平⽅和再做平均作为选择最优模型结构的依据。
特别的K取N，就是留⼀法（leave one out）。
这三个名词在机器学习领域的⽂章中极其常见，但很多⼈对他们的概念并不是特别清楚，尤其是后两个经常被⼈混⽤。
Ripley, B.D（1996）在他的经典专著Pattern Recognition and Neural Networks中给出了这三个词的定义。
Training set: A set of examples used for learning, which is to fit the parameters [i.e., weights] of the classifier.Validation set: A set of examples used to tune the parameters [i.e., architecture, not weights] of a classifier, for example to choose the number of hidden units in a neural network.Test set: A set of examples used only to assess the performance [generalization] of a fully specified classifier.显然，training set是⽤来训练模型或确定模型参数的，如ANN中权值等； validation set是⽤来做模型选择（model selection），即做模型的最终优化及确定的，如ANN的结构；⽽ test set则纯粹是为了测试已经训练好的模型的推⼴能⼒。
I.J. Information Engineering and Electronic Business, 2019, 1, 19-26Published Online January 2019 in MECS (/)DOI: 10.5815/ijieeb.2019.01.03An Approach for Forecast Prediction in Data Analytics Field by Tableau SoftwareBibhudutta JenaEmail: bibhudutta93@Received: 13 December 2017; Accepted: 18 October 2018; Published: 08 January 2019Abstract—The current era is generally treated as the era of data, Users of computer are gradually increasing day by day and vast amount of data is generated from multiple domains such as healthcare- domain, Business related domains etc. The terminology Business Intelligence (BI) generally refers different technologies, applications and practices used for the collection, integration, analysis, and presentation of information of business related domain. The main motive for Business intelligence and analytics are to help in decision making process and to enhance the profit of the organisation. Various business related tools are used to analyze & visualize different types of data which are generated frequently. Tableau prepared its mark on the Field of BI by being one of the first companies to permit business customers the ability to achieve equitably arduous data visualization in a very interesting, drag and drop manner. Tableau will enhance decision making, add operational awareness, and increase performance throughout the organization The presented paper describes different tools used for business intelligence field and provides a depth knowledge regarding the tableau tool. It also describes why tableau is widely used for data visualization purpose in different organization day by day. The main aim of this paper is to describe how easily forecasting and analysis can be done by using this tool ,this paper has explained how easily prediction can be done through tableau by taking the dataset of a superstore and predict the forthcoming sales and profit for the next four quarters of the forthcoming year. In the collected dataset sales and profit details of different categories of goods are given and by using the forecasting method in tableau platform these two measures are calculated for the forthcoming year and represented in a fruitful way. Finally, the paper has compared all the framework used for business intelligence and analytics on the basis of various parameters such as complexity, speed etc.Index Terms—Big Data, Business Intelligence, Data Visualization, Tableau, QlickVieew.I.I NTRODUCTIONDue to the increased market competition increased data management and analysis has landed as in an era that requires further optimization data management and analysis. Big data technologies like apache HADOOP provide a frame work for parallel data processing and generation of analyzed results .MAPREDUCE method is used for analysis of data using various data analysis algorithms like clustering, fragmentation and aggregation. (BI) generally refers different technologies, applications and practices used for the collection, integration, analysis, and presentation of information of business related domain[1]. The main motive for Business intelligence and analytics are to help in decision making process and to enhance the profit of the organisation. Various business related tools are used to analyze & visualize different types of data which are generated frequently. Data is one of the most important and vital aspect of different activities in today's world. Now -a -days the modern civilization gradually more dependent on the age of information which is concentrated in computers, So the Society is becoming more dependent on computers and technology for functioning in every- day life .As the number of users are gradually increasing day by day, So the data generated by them is also enlarging gradually[2].The main aim of this paper is to describe how easily forecasting and analysis can be done by using this tool ,this paper has explained how easily prediction can be done through tableau by taking the dataset of a superstore and predict the forthcoming sales and profit for the next four quarters of the forthcoming year. The reminder of this paper is organized as. Section II presents a literature survey on various approaches and tools used for business intelligence and analysis purpose. Section III presents a brief idea about Tableau tools and its application. The motivation for this work is laid out in Section II. Section IV represents the Implementation part for the forecast analysis of sample retail store data Finally, Section V concludes the paper.II.L ITERATURE S URVEYTableau presently has the largest customer domain. When businesses organization are focusing for a BI solution, it’s always their first choice. In the below section the paper has described different BI tools that are currently used in the business market for enhancing the business strategy. Sisense is one of the leaders in the market of BI and a winner of the Best Business Intelligence Software Award given by Finances Online [3][4], It was considered as one of the most adopted business, software review platforms. The users of thistool will also enjoy effectively by using the in-chip technology in a database that operates the data much more faste than the existing traditional systems. BIRT project is a flexible, Standalone, and 100% pure Java reporting tool for creating and presenting reports in opposition to data sources varying from typical business relational databases, to XML data sources, to in-memory Java objects. It is originated as a top-level project integrating with the Eclipse Foundation. Similarly icCube is a SaaS end to end BI platform, particularly to be ingrained in to the developed application[5][6]. It amalgamates smoothly with any application due to on-the-fly-authentication and authorization ability. icCube can be treated as a dream for any software developer who wants to present predicated dashboards or a solid web based self service BI solution, to their end-users. It is not Cost effective. Business Optimization Software Domos imports together the customers, the data, and the acumen business users demand to present a detailed view of what’s happening in the organization[7][8]. The Jaspersoft Business Intelligence Suite provides different ways for end customers to initiate interactive analysis. In the case of casual user, this might include simply changing a filter setting on a report to present various slice of data.This tool is generally helpful for data analyst to writing powerful, multi-dimensional expressions. Microsoft also provides Business Intelligence platform comprises of Analysis Services, Integration Services, Master Data Services, Reporting Services, and several client applications used for initiating or operating with analytical data. But this software is not able to make the dashboard attractive like other BI tools. This data is often travelled across various databases in multiple locations with different versions of database software. .Pentaho platform provides an easier way for preparing and blending any data and adds a spectrum of tools to easily analyze, visualize, explore, report and predict[9]. In the above section the paper has discussed some business intelligence tools which are earlier used for the business intelligence and business analysis processes. In the next section the paper has described how tableau is helpful for business intelligence and analysis purpose which plays an important role for decision making processes in the organisation.III.T OOL S TUDYTableau prepared its mark on the Field of BI by being one of the first companies to permit business customers the ability to achieve equitably arduous data visualization in a very interesting, drag and drop manner. Tableau’s data virtualization is head and shoulders above what traditional BI vendors offer[10]. Tableau occupied its mark on the world of BI by being one of the first companies to give business customers the ability to operate fairly complex data visualization in a very intuitive, drag and drop manner. In this platform visualization and analysis can be done without having proper knowledge in programming domain, The output analyzed results can be shown in an effective manner. In the below section the paper has described the how smoothly the forecasting process can be done by tableau tool by taking a business test case. Tableau clearly and beautifully visualizes your data. Tableau has the functionality to present reports on supreme large sets of data without enormously affecting network performance. someone can easily go from novice to expert level with proper assistance through this tool.IV.I MPLEMENTATIONThis section describes how the data is processed in tableau platform and how forecasting can be done with the collected dataset. The data which are collected for processing are from a sample store present in USA which comprises of different attributes or fields , In the below figure describes the excel format of data that is taken for processing in tableau platform. There are multiple ways to import the data set in tableau software, One of the biggest leverage of tableau software is, it allows the user to import the data in portable document format and perform different data analytic operation by utilizing the data.Tableau’s data virtualization is leader and shoulders above what conventional BI vendors offer. Tableau done its mark on the world of BI by being one of the first companies to give business users the propensity to perform fairly complex data visualization in a very visceral drag and drop manner.Tableau assimilates with most data types and provides out of the box integration with a variety of big data platforms, comprising Hadoop. Tableau assimilates with R, the business intelligence statistical language many data scientists use to dive deep into BI for statistical and advanced analytics,Tableau has a ample amount of partner and consultant base, as well as exhaustive online resources including guides, online training and forums. Plus, the Tableau community is well-known enthusiastic and engaging., Like most BI vendors, Tableau provides different software licensing options. There’s a nonpaid version of Tableau for personal use, but its scope is limited. To download Tableau to your desktop, you’ll pay in the range of $1000 and $2000 per year. Online or hosted access to Tableau costs $500 per person, per year. In the below paragraph the paper has described some cons of tableau server and toolHence, lot of organizations will need Tableau server. Based on what companies that have considered Tableau (or have purchased it) tell us, an entry-level server license is about $1000 per user (with a 10-user minimum).If data is in Excel or a CSV file format, you can upload it to Tableau and perform different data analytical operations Tableau occupied its mark on the world of BI by being one of the first companies to give business customers the ability to operate fairly complex data visualization in a very intuitive, drag and drop manner.. But, if you want to interact to a database, a developer skilled in SQL will have to create the SQL query to retrieve the dataset.With the Tableau server, there is no term of versioning. You build your reports and dashboards through your desktop and then publish them to the server. Once they’republished, there’s no alternate w ay to retrieve previous versions- once you overwrite there’s no pulling back options is available. , In the below figure [1] describes the excel format of data that is taken for processing in tableau platform.Fig.1. Original Data source in excel sheetThe below figure [2] describes the process of importing the original data source which is present in excel format to tableau-platform.Fig.2. Importing data set in to tableau frameworkAfter successfully importing the data in to tableau platform the data is divided in to two parts the numeric data are generally come in to the measure section and rest other data come to the dimension section. This division of data is generally done by the tableau itself. The a division of data in to dimension and measure section which is done by tableau internally, In the below figure[3] andFigure [4] the paper describes the data that is present on the original data source ,In the original data set profit and sale information about various, categories of products are present up to year 2017 and the paper has forecasted the profit and sale information on the forthcoming quarters of 2018.Fig.3. Profit Data Retrieve From Tableau ToolFig 4.Sales Data retrieve from tableau toolIn the below section the paper has described the estimated profit of different categories of product in forthcoming quarters. In Figure [5] the estimated profit for Furniture Category for forthcoming quarters are calculated ,In similar manner Figure[ 6] provides the estimate calculation of technology category product and Figure [7] represents the estimated calculation Office supplies product category, In tableau the forecast calculation is done on the basis of import data .Fig.5. Estimated Profit Calculation For Furniture CategoryFig.6. Estimated Profit Calculation For Technology CategoryFig.7. Estimated Profit Calculation For Office Supplies CategoryIn the below section the paper has described the estimated Sales of different categories of product in forthcoming quarters. In Figure [8] the estimated sale for Furniture Category for forthcoming quarters are calculated ,In similar manner Figure[ 9] provides the estimate sale calculation of technology category product and Figure [10] represents the estimated sale calculation of Office supplies product category, In tableau the forecast calculation is done on the basis of currently available sales data of different quarters. .Fig.8. Estimated Sale Calculation For Furniture CategoryFig.9. Estimated Sale Calculation For Technology CategoryFig.10. Estimated Sale Calculation For Office Supplies CategoryV.C ONCLUSIONDue to the increased market competition and increased data management an era has reached that needs more optimization technique to analysis the data. Big data technologies such as HADOOP proposes a framework for processing the data parallel y and after processing the data different visualization tools are used for representing the analyzed result in an effective manner which plays an important for decision making process in different business organizations[11]. Tableau will enhance decision making, add operational awareness, and increase performance throughout the organization[12]. The presented paper describes different tools used for business intelligence field and provides a depth knowledge regarding the tableau tool. Data visualization plays an important role for the business analytics in order to do different operation with data. Tableau analytics is one of the easiest and most powerful analytics tools today’s era[13].Tableau can assist to assemble sense of different types of data coming in from multiple sources and blend it into one unique platform so that everybody can make sense of it and utilize it[14].The Tableau Dashboard is an important feature of the Tableau Product that lets you allow to create unique data stories, It is constructed from the ground up for people who do not have any technical skills or coding experience for that matter. So anything can be done by anybody without any prior set of skills. This paper has explained the forecast methodologies in tableau framework and practically done the forecast by taking the dataset from business domain. There are many various types of visualization options available in Tableau which enhance the user experience. Also, Tableau is very easy to learn compared to Python, Business Objects and Domo, anyone without having knowledge of coding can easily learn Tableau. Tableau can operate millions of rows of data with ease. Various types of visualization can be initiated with a huge amount of data without impacting the performance of the dashboards. Also, there is an option available in Tableau where the user can make and can “live” to connections to different data sources like SQL etc and can perform multiple data manipulation operation [16] with the live data. Tableau Dashboard has a marvellous reporting feature that allows you to create personalize dashboard specifically for a convinced device such as a mobile phone or laptop. Tableau automatically figure out which device is the user is viewing the report on and do adjustments to ensure that the right report is delivered to the right device.Tableau has done a great job climb its way to the top of data visualization tools. So, Tableau has spent more than six years as a leader. However, with the increasing interest in data science, artificial intelligence, and machine learning, Tableau may be left behind if it doesn’t innovate quickly. Tableau does not support the aspect of automatic exhilarating of the reports with the help of scheduling. There is no other choice of scheduling in Tableau. Therefore, there is always some manual effort required when users need to update the data in the back-end. Tableau is not entirely an open tool. Just like other tools like Power BI, developers can create custom visuals that can be easily imported Tableau. So, any new visuals need to be recreated instead of imported. Tableau’s parameters are constant and always unique value can be preferred using a parameter. Whenever the data gets changed, these parameters need to be updated manually every time.VI.F UTURE S COPETableau is strictly a visualization tool. Tableau Desktop only permits you to perform very basic preprocessing, Which includes joining and blendingdata[17]. Also, you have the competence to change data types. In an ideal world, most data would be exported in perfect tables. However, data cleansing is a necessary step. In most cases, an analyst needs to build a data model with reappearing to format the data. This needs a tool such as Altyrex, Power BI, Python or even Excel to operate data prior to loading[18]. In 2018, introduced their own data preparation tool called Tableau Prep, Different researches are going on for optimizing the execution time and reduce the elapsed time for performing data analytics operation in tableau platform.R EFERENCES[1]Raghupathi W: Data Mining in Health Care. In HealthcareInformatics: Improving Efficiency and Productivity.Edited by Kudyba S. Taylor & Francis; 2010:211–223. [2] B. Jena, M. K. Gourisaria, S. S. Rautaray, and M. Pandey,\A surveywork on optimization techniques utilizing map reduce framework in hadoop cluster," International Journal of Intelligent Systems and Appli- cations, vol. 9, no. 4, p. 61, 2017.[3] B. Jena, M. K. Gourisaria, S. S. Rautaray and M. Pandey,"Name node performance enlarging by aggregator based HADOOP framework," 2017 International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), Palladam, 2017, pp. 112-116.[4] B. Jena, M. K. Gourisaria, S. S. Rautaray and M. Pandey,"Improvising name node performance by aggregator aided HADOOP framework," 2016 International Conference on Control, Instrumentation, Communication and Computational Technologies (ICCICCT), Kumaracoil, 2016, pp. 382-388.[5]Nages h HR, Guru Prasad “High PerformanceComputation of Big Data: Performance Optimization Approach towards a Parallel Frequent Item Set Mining Algorithm for Transaction Data based on Hadoop MapReduce Framework” International Journal of Intelligent Systems and Applications(IJISA), Vol.9, No.1, pp.75-84, 2017. DOI: 10.5815/ijisa.2017.01.08.[6]Siddharth S Rautaray, and Manjusha Pandey, “Single andMultiple Hand Gesture Recognition Systems: A Comparative Analysis”, I.J. Intelligent Systems and Applications, 6 (11), 57-65, 2014.[7]Rath Jairak, Prasong Praneetpolgrang, NivetChirawichitchai,"A Roadmap for Establishing Trust Management Strategy in E-Commerce Services Using Quality Based Assessment", IJIEEB, vol.6, no.5, pp.1-9, 2014. DOI: 10.5815/ijieeb.2014.05.01.[8]Jeffrey Dean and Sanjay Ghemawat , " Map Reduce:Simplified Data Processing on Large Clusters", IEEE Micro, 23(2):2228, April 2005.[9]Esma Yildirim, Engin Arslan, Jangyoung Kim, TevfikKosar. "Application-Level Optimization of Big Data Transfers through Pipelining, Parallelism and Concurrency", IEEE Transactions on Cloud Computing,2016[10]J. Archenaa, E.A. Mary Anita,” A Survey of Big DataAnalytics in Healthcare and Government”, Procedia Computer Science, Elsevier, Volume 50, 2015, Pages 408–413,Big Data, Cloud .[11]Gunjan Varshney1, D. S. Chauhan2, M. P. Dave,”Evaluation of Power Quality Issues in Grid Connected PV Systems”, International Journal of Electrical and Computer Engineering (IJECE), Vol. 6, No. 4, August 2016, pp. 1412~1420. [12]Ziv J., Lempel A., “A Universal Algorithm for SequentialData Compression,” IEEE Transactions on Information Theory, Vol. 23, No. 3, pp. 337-343.[13]N.E. Ayat, M. Cheriet, C.Y. Suen ,“Automatic modelselection for the optimization of SVM kernels,” Artificial intelligence in medicine, vol. 38, no.10, pp. 1733-1745, 2005.[14]Hamid Bagheri , Abdusalam Abdullah Shaltooki., " BigData: Challenges, Opportunities and Cloud Based Solutions", International Journal of Electrical and Computer Engineering (IJECE),2015.[15]Sabyasachi Chakraborty, Kashyap Barua, ManjushaPandey, Siddharth Rautaray," A Proposal for High Availability of HDFS Architecture based on Threshold Limit and Saturation Limit of the Namenode", International Journal of Information Engineering and Electronic Business(IJIEEB), Vol.9, No.6, pp. 27-34, 2017. DOI: 10.5815/ijieeb.2017.06.04.[16]Mayank Bhushan , Monica Singh , Sumit K Yadav ,"Big Data query optimization by using Locality Sensitive Bloom Filter ",IJCT, 2015.[17]Liu, Yunxiang, and Jiongjun Du. "Parameter Optimizationof the SVM for Big Data", 2015 8th International Symposium on Computational Intelligence and Design (ISCID), 2015.[18]Lanchao Liu and Zhu Han , " Multi-Block ADMM forBig Data Optimization in Smart Grid " , IEEE, 2015. [19]Al-Madi, Nailah, Ibrahim Aljarah, and Simone A. Ludwig."Parallel glowworm swarm optimization clustering algorithm based on MapReduce", 2014 IEEE Symposium on Swarm Intelligence, 2014.[20] A. Ramaprasath, K. Hariharan, A. Srinivasan, “CacheCoherency Algorithm to Optimize Bandwidth in Mobile Networks”, Springer Verlag, Lecture Notes in Electrical Engineering, Networks and Communications, Chapter 24, Volume 284, 2014, pp 297-305.[21]Ziv J., Lempel A., “A Universal Algorithm for SequentialData Compression,” IEEE Transactions on Information Theory, Vol. 23, No. 3, pp. 337-343[22] E. Yildirim, J. Kim, and T. Kosar, “Optimizing the samplesize for a cloud-hosted data schedulin g service,” in Proc.2nd Int. Workshop Cloud Computing. Sci. Appl., 2012. [23]Anitha P, Malini M. Patil," A Review on Data Analyticsfor Supply Chain Management: A Case study", International Journal of Information Engineering and Electronic Business(IJIEEB), Vol.10, No.5, pp. 30-39, 2018. DOI: 10.5815/ijieeb.2018.05.05[24]Abdus Satter, Nabil Ibtehaz," A Regression based SensorData Prediction Technique to Analyze Data Trustworthiness in Cyber-Physical System", International Journal of Information Engineering and Electronic Business(IJIEEB), Vol.10, No.3, pp. 15-22, 2018. DOI:10.5815/ijieeb.2018.03.03Author’s ProfileBibhudutta Jena, is a CSI AccreditedStudent. Completed M. Tech (ComputerScience and Engineering) at the School ofComputer Engineering, KIIT University,Bhubaneswar. His areas of interest DataAnalytics ,Data mining,Data visualizationetc .He has already published more than10 research papers in big data and analytics domain,He is currently pursuing as ansoftwareengineer in MPHASIS Limited. He can be reached atbibhudutta93@.How to cite this paper: Bibhudutta Jena, "An Approach for Forecast Prediction in Data Analytics Field by Tableau Software", International Journal of Information Engineering and Electronic Business(IJIEEB), Vol.11, No.1, pp. 19-26, 2019. DOI: 10.5815/ijieeb.2019.01.03。
Data Visualization StrategiesData visualization is a crucial aspect of data analysis, as it allows for the presentation of complex information in a visual format that is easy to understand and interpret. There are various strategies and techniques that can be employed to effectively visualize data, each with its own strengths and limitations. In this response, we will explore some of the most commonly used data visualization strategies, their applications, and the benefits they offer to data analysts and decision-makers.One of the most popular data visualization strategies is the use of charts and graphs. Charts and graphs are effective in presenting numerical data in a visual format, making it easier for the audience to identify patterns, trends, and outliers. Common types of charts and graphs include bar charts, line graphs, pie charts, and scatter plots. Each type of chart or graph is suitable for different types of data and can be used to convey different messages. For example, a bar chart is useful for comparing the values of different categories, while a line graph is effective in showing trends over time.Another important data visualization strategy is the use of maps. Maps are particularly useful for visualizing geographical data, such as population distribution, sales by region, or the spread of diseases. By overlaying data on a map, analysts can easily identify spatial patterns and make informed decisions based on the geographic distribution of the data. Geographic information system (GIS) software is commonly used to create and analyze maps, allowing for the visualization of spatial data in a variety of formats, such as choropleth maps, heat maps, and point maps.In addition to charts, graphs, and maps, data analysts can also utilize infographics as a data visualization strategy. Infographics are a visually appealing way to present complex information, combining text, images, and graphics to convey a message or tell a story. Infographics are particularly effective in summarizing large amounts of data and making it more accessible to a wider audience. By using a combination of visuals and concise text, infographics can help viewers quickly grasp the key insights from the data without being overwhelmed by too much information.Furthermore, interactive data visualization is becoming increasingly popular as a strategy for engaging audiences and allowing them to explore data in a more dynamic way. Interactive visualizations allow users to interact with the data, such as by hovering over data points for more details, filtering the data based on specific criteria, or zooming in and out of a visual representation. This level of interactivity can enhance the user experience and empower individuals to discover insights that may not be immediately apparent in static visualizations.Moreover, storytelling through data visualization is an emerging strategy that focuses on using visuals to convey a narrative or make a compelling argument. By structuring data visualizations in a way that tells a story, analysts can guide their audience through the data, leading them to a specific conclusion or insight. Storytelling through data visualization often involves the use of a sequence of visualizations that build upon each other to communicate a coherent message, engaging the audience and helping them understand the significance of the data.Lastly, the use of dashboards as a data visualization strategy is beneficial for providing a comprehensive overview of key performance indicators, metrics, and trends. Dashboards typically consist of multiple visualizations and data displays that are organized in a single interface, allowing users to monitor and analyze data in real-time. Dashboards are commonly used in business intelligence and analytics to track progress towards goals, identify areas for improvement, and make data-driven decisions. The interactive nature of dashboards also enables users to drill down into specific data points and gain deeper insights into the underlying data.In conclusion, data visualization is a powerful tool for making sense of complex data and communicating insights effectively. By employing various data visualization strategies, such as charts, graphs, maps, infographics, interactive visualizations, storytelling, and dashboards, data analysts can present data in a compelling and informative manner. Each strategy offers unique benefits and can be applied to different types of data and analytical objectives. Ultimately, the choice of data visualization strategy depends on the nature of the data, the audience, and the specific insights that need to be conveyed. As technologycontinues to advance, new data visualization strategies and techniques will undoubtedly emerge, further enhancing the ability to explore and understand data in meaningful ways.。
