Abstractions for Software Architecture and Tools to Support Them
Mary Shaw, Robert DeLine, Daniel V. Klein, Theodore L. Ross, David M. Young, Gregory Zelesnik
Computer Science Department Carnegie Mellon University Pittsburgh PA and various other current affiliations1
Version of March 8, 1995
Abstract
Architectures for software use rich abstractions and idioms to describe system components, the nature of interactions among the components, and the patterns that guide the composition of components into systems. These abstractions are higher-level than the elements usually supported by programming languages and tools. They capture packaging and interaction issues as well as computational functionality. Well-established (if informal) patterns guide architectural design of systems. We sketch a model for defining architectures and present an implementation of the basic level of that model. Our purpose is to support the abstractions used in practice by software designers. The implementation provides a testbed for experiments with a variety of system construction mechanisms. It distinguishes among different types of components and different ways these components can interact. It supports abstract interactions such as data flow and scheduling on the same footing as simple procedure call. It can express and check appropriate compatibility restrictions and configuration constraints. It accepts existing code as components, incurring no runtime overhead after initialization. It allows easy incorporation of specifications and associated analysis tools developed elsewhere. The implementation provides a base for extending the notation and validating the model. Keywords: Software architecture, architecture description language, software system organization, architectural abstraction, software engineering
1Mary
Shaw holds a joint appointment with the Software Engineering Institute, Carnegie Mellon University. Daniel V. Klein is currently with LoneWolf Systems, Pittsburgh PA. Theodore L. Ross is currently with Digital Equipment Corporation, Littleton MA. David M. Young is currently with Madeira Software, Inc., Beverly MA. Electronic mail contact address: mary.shaw@https://www.doczj.com/doc/55148555.html,
Mary Shaw Abstractions for Software Architecture and Tools to Support Them 1
Table of Contents 1 . Introduction 2 . Model and Notation
2.1. 2.2. 2.3. 2.4. 3.1. Components and Connectors Abstraction and Encapsulation Types and Type Checking Accommodating Analysis Tools Semantics
15 18 21 22 23 25 27
3 8
9 11 12 13
3 . UniCon: Language for Universal Connector Support
3.1.1. Components 3.1.1.1. Built-in Component Types 3.1.1.2. Implementation of Primitive Components 3.1.1.3. Implementation of Composite Components 3.1.2. Connectors 3.1.2.1. Built-in Connector Types 3.1.2.2. Implementation of Primitive Connectors
13
14
3.2. 3.3.
Graphical Notation Textual Notation
Major Constructs Primitive Components Composite Components Primitive Connectors Property Lists 29 30 31 33 34
28 29
3.3.1. 3.3.2. 3.3.3. 3.3.4. 3.3.5.
3.4. 3.5. 4.1. 4.2. 4.3. 4.4. 5.1. 5.2. 5.3.
Populating the Space of Elements Incorporating Analysis Tools Procedure Call and Global Data Access Unix Pipes and Files Remote Procedure Call Real-Time Scheduling Experience Performance Conclusion
34 35
4 . Implementation
39
39 40 41 42
5 . Experience and Analysis
43
43 45 46
Appendix: References
48 48
Mary Shaw
Abstractions for Software Architecture and Tools to Support Them
2
1.
Introduction
Software engineers often describe the “architectures” of their systems. These descriptions address high-level aspects of the systems such as the overall organization, the decomposition into components, the assignment of functionality to components, and the way the components interact. These architectural descriptions often use box-and-line diagrams and phrases such as “pipe-and-filter system” and “client-server model”. While architectural descriptions use high-level abstractions, the corresponding implementations are written in conventional programming languages. However, composing a system from subsystems is a substantially different activity from programming the underlying algorithms and data structures. Hardware designers confront the same situation; they recognize a number of distinct design levels, each with its own design issues, models, notations, componentry, and analysis techniques [Bell & Newell 71]. In the same way, different levels of software design require different kinds of components, different ways of composing components, different design issues, and different kinds of reasoning. The vocabulary gap between requirements and programming is substantial. Filling the gap requires better models and notations for the intermediate step. This is the goal of the emerging field of software architecture. The architecture of a software system defines that system in terms of components and of interactions among those components. In addition to specifying the structure and topology of the system, the architecture shows the intended correspondence between the system requirements and elements of the constructed system. It can additionally address system-level properties such as capacity, throughput, consistency, and component compatibility. Architectural models clarify structural and semantic differences among components and interactions. Architectural definitions can be composed to define larger systems. Elements are defined independently so they can be re-used in different contexts. The architecture establishes specifications for individual elements to be written in a conventional programming language. A number of commonly-used patterns, or idioms, are in widespread informal use; these architectural styles can be captured as general templates for families of related systems. This holds particular promise for domain-specific systems. [Garlan & Shaw 93, Mettala & Graham 92, Perry & Wolf 92] Our primary considerations here are supporting architectural abstractions, localizing and codifying the ways components interact, and distinguishing among the various packagings of components that require different forms of interaction. Our focus is largely pragmatic. Our first concern has been to identify, classify, and support a variety of components and their connections. We will refine the notation and develop a formal base over time. A sound basis for software architecture promises benefits for both development and maintenance. Design should benefit from improved abstractions, notations, and analysis. Architectural definitions should help provide good specifications for programming activities. Since the architectural definition also serves as the specification for system build, the specifications and code will be less likely to diverge. Maintenance should benefit in two ways. First, about half of maintenance effort is dedicated to understanding the system in preparation for making changes; explicit definition of the architecture should reduce this cost. Second, system architectures degrade over time; carrying the architectural definition into maintenance should reduce this tendency toward degradation. Additional development benefits should accrue through better information to guide software reuse. Reuse is currently impeded by differences in component packaging. For example, a given funcMary Shaw Abstractions for Software Architecture and Tools to Support Them 3
tionality might be packaged as a procedure, a communicating process, an object, or a filter; it might interact with other components by calls, message-passing, method invocation, or shared data. These differences in packaging are recognized only informally and are not supported by programming languages and tools; neither formal nor informal guidance shows when and how to use them. As a result, it is often unclear whether components with compatible functionality will actually be able to interact properly. Documenting a system’s structure and properties in a rigorous way has obvious advantages for maintenance. Much of the time spent on maintenance goes to understanding the existing code; this effort should be reduced substantially if the original design structure is captured clearly and explicitly. In addition, retaining the designer’s intentions about system organization should help maintainers preserve the system’s design integrity. A growing community of researchers is focusing on software architecture. There has been longstanding interest in particular classes of architectures such as objects (focused by the OOPSLA conference), pipelines (focused by the USENIX conference), and client-server systems (a subject of intense attention in the commercial data processing arena). Early attention to the variety of idiomatic patterns of system organization and the possibility of organizing this knowledge systematically [Shaw 88, Perry & Wolf 92, Boehm & Scherlis 92, Garlan & Shaw 93] have led to enough activity to sustain workshops on architectural design [Barstow & Wolf 93], software design with strong participation from architecture [Lamb 95], patterns for design [PLoP 94], and interface definition languages [Wing 94]. Software architecture emerged as a key theme at a recent workshop on directions in software engineering [Habermann & Tichy 92]. A major effort to gain design power for specific kinds of problems addresses domain-specific software architectures (DSSAs) in several specific domains [Mettala & Graham 92, Hayes-Roth & Tracz 93]. Formalizations are beginning to emerge [Allen & Garlan 94a,b]. As a natural consequence, languages for describing architectures are emerging [Dean & Cordy 93, Rapide 93]. The need for a notation to describe how subsystems written in typical programming languages connect to form larger systems is not a new concern. In 1975, DeRemer and Kron [DeRemer & Kron 76] argued that creating program modules and connecting them together to form larger structures were distinct design efforts; they created the first module interconnection language (MIL) to support the connection effort. In an MIL notation, modules import and export resources, which are named elements such as type definitions, constants, variables, and functions. Compilers for MILs ensure system integrity with intermodule type checking: they check that if one module uses a resource that another provides, the types of the resources match; that if a module declares it provides a resource, it actually does; that if a module uses a resource, it has access to that resource; and so on. Since DeRemer and Kron’s MIL, MILs have been developed for specific languages, like Mesa [Mitchell et al 79] and Ada [Campos & Estrin 78], and have provided a base from which to support software construction [Thomas 76], version control [Cooprider 79], system families [Tichy 79], and dynamic configuration [Magee et al 89]. Enough examples are available to develop models of the design space [Perry 87, Prieto-Diaz & Neighbors 86]. These early module interconnection languages require considerable prior agreement between the developers of different modules. For example, they assume that simple name matching can be used to infer inter-module interaction, that all modules are written in the same language, that all modules are available during system construction, and that module interfaces describe the other modules with which they interact. Newer work has begun to soften these restrictions. In the Darwin language, modules can be dynamically instantiated and bound at runtime [Magee et al 93]. Polygen [Callahan & Purtilo 91] augments a module interconnection language with an inference engine that deduces from a user-defined set of rules how (or whether) a system can be integrated from set of modules.
Mary Shaw Abstractions for Software Architecture and Tools to Support Them 4
These modules can be implemented in multiple programming languages, and the machinery needed to connect them can be richer than the usual procedure linkage, for example, a software bus [Purtilo 90]. This kind of system requires expanding the notion of a MIL to include specifics about a module's implementation, such as its programming language, its hardware/operating system platform, and the communication media needed to access it; the resulting richer notation has been termed a module interconnection formalism (MIF). To build truly composable systems we must allow flexible, high-level connections between existing systems in ways not foreseen by their original developers. Essentially independently, developers of “open” software products have designed interchange representations such as PICT (line drawings), RTF (formatted text), SYLK (spreadsheet layouts), and MIF (formatted text) to allow distinct products to interact by data interchange. Although these were originally static, newer developments such as CORBA, OLE, and OpenDoc (all for objects) support dynamic sharing. Systems often exhibit an overall style that follows a well-recognized, though informal, idiomatic pattern. Garlan and Shaw survey half a dozen of these patterns and illustrate their use in case studies [Garlan & Shaw 93]. They identify pipes and filters, data abstractions or objects, implicit invocation, hierarchical layers, repositories, and interpreters as a useful (though incomplete) set of well-known patterns, or styles, of system organization. These styles differ both in the kinds of components they use and in the way those components interact with each other. As advocates of various of these architectures explain, adherence to the rules of the style enhances both software development and subsequent maintenance.2 The rules of the style usually restrict how to package components—e.g., as procedures, objects, or filters. As a result, components may not be usable in all styles; code may not reusable because its interface makes incompatible assumptions. In Unix, for example, the functionality of “sort” is available both in the form of a filter and in the form of a procedure. The remainder of this paper is organized as follows: Section 2 introduces our model of software architecture and its notation; Section 3 describes UniCon, an initial language for implementing the model; Section 4 highlights its implementation; Section 5 describes our experience with UniCon.
File Pipe
req-data
Filter
merge
sorter
caps
shifter
Figure 1. Pipe-and-filter example: the KWIC indexer.
We will use three examples throughout the paper. Although architecture diagrams often do not make visual distinctions among the interactions they depict, here we use different markings for different types of connections. The first example, shown in Figure 1, is a Unix-style pipe-and-filter system built from both filters and files, with the wrinkle that the pipeline merges two streams—a configuration that is difficult or impossible to describe in most shells. The system implements a KWIC (keyword in context) indexer; we have used a very similar task as a class exercise in a software ar2However,
the advocates often neglect to mention that different architectures are appropriate in different situations. Choosing the most appropriate architecture for a given problem (or domain) remains an open problem.
Abstractions for Software Architecture and Tools to Support Them 5
Mary Shaw
chitecture class [Garlan & Shaw 94]. The challenge of this example is to make complex topologies as easy to describe as simple ones.
gatherer
sorter
reverser
Module Implements a Filter
rev
Procedure Call and Shared Data
stk
lib
Module
Figure 2. Heterogeneous implementation of a pipeline using both pipes and procedures. The upper diagram is the high-level view of the system, a simple pipeline. The lower diagram shows the implementation of the right-most component in the upper diagram.
The second example, shown in Figure 2, combines a pipe-and-filter architecture with a conventional procedural implementation of one of the filters. The challenge of this example is to compose architectural descriptions and to establish the correspondence between the abstraction of a pipeline and its implementation as calls on system procedures.
Real-Time Scheduler
client
server
Remote Procedure Call Schedulable Process
Figure 3. Real-time client-server system with two schedulable tasks sharing a computing resource. The tasks also interact via remote procedure call.
Mary Shaw
Abstractions for Software Architecture and Tools to Support Them
6
The third example, shown in Figure 3, involves coordinating real-time tasks. A simple periodic realtime system has a number of tasks that must run on specified schedules. More challenging scheduling problems arise when tasks interact. We use an example with two schedulable tasks that interact through remote procedure calls. Based on a system timer, the client must periodically perform computations which involve calling upon services provided by the server; correctness requires that these events take place at specific times. The challenge of this example is to incorporate an external analysis tool that determines the legality of the configuration (especially to make guarantees about schedulability) and to convert the code for the tasks into schedulable processes that run on a real-time operating system.
2.
Model and Notation
This section describes an informal model for an architectural description language. The language is intended to aid designers in first defining software architectures using abstractions they find useful and then making a smooth transition to code. Our long-term objective is to fully elaborate this model and to support it with notation and tools. Currently, we are more strongly motivated by practical utility of the model than by formal foundations. At present, the model provides a framework for understanding our initial implementation, which is described in Section 3. The model addresses several issues in novel ways: ? It supports abstraction idioms commonly used by designers—for example, explicitly distinguishing different types of elements and providing type-specific analysis support. ? It specifies packaging properties as well as functional properties of components—for example, distinguishing clearly between functionality delivered in the form of a filter from functionality delivered in the form of a procedure. ? It provides an explicit, localized home, called a connector, for information about the rules for component interactions, such as protocols, interchange representations, and specifications of data formats for communication. ? It defines an abstraction function to map from code or lower-level constructs to higher-level constructs. This is similar to Hoare’s technique for abstract data types [Hoare 72]. ? It is open with respect to externally-developed construction and analysis tools. It supports collection and delivery of relevant information to tool and return of results from the tool. Software system composition involves different tasks from writing modules: the system designer defines roles and relationships rather than algorithms and data structures. These concerns are sufficiently different to require a separate language. The architectural language must support system configuration, independence of entities (hence reusability), abstraction, and analysis of properties ranging from functionality to security and reliability [Shaw & Garlan 93]. The model must be supported by a notation.
2.1.
Components and Connectors
Systems are composed from identifiable components and connectors of various distinct types. The components interact in identifiable, distinct ways. Components roughly correspond to compilation units of conventional programming languages and other user-level objects such as files. Connectors mediate interactions among components; that is, they establish the rules that govern component interaction and specify any auxiliary implementation mechanism required. Connectors do not in general correspond individually to compilation units; they manifest themselves as table entries, buffers,
Mary Shaw Abstractions for Software Architecture and Tools to Support Them 7
instructions to a linker, dynamic data structures, sequences of system calls embedded in code, initialization parameters, servers that support multiple independent connections, and the like. During system design, it is important to work with good abstractions for interaction without concern for whether their implementations are localized. It is helpful to think of the connector as defining a set of roles that specific named entities of the components must play. Our model thus describes software systems in terms of two kinds of distinct, identifiable elements: components and connectors. Each of the two elements has a type, a specification, and an implementation. The specification defines the units of association used in system composition; the implementation can be primitive or composite. Figure 4 suggests the essential character of the model.
Element Specification
Type Unit of association
Component Interface
Component Type Player
Connector Protocol
Connector Type Role
Implementation
Implementation
Implementation
Figure 4: Gross structure of an architecture language.
Components are the locus of computation and state. Each component has an interface specification that defines its properties. These properties include the component's type or subtype (e.g. filter, process, server, data storage), functionality, guarantees about global invariants, performance characteristics, and so on. The specific named entities visible in a component’s interface are its players. The interface includes the signature, functionality, and interaction properties of its players. Connectors are the locus of definition for relations among components. They mediate interactions but are not “things” to be “hooked up;” rather, they provide the rules for hooking-up. Each connector has a protocol specification that defines its properties. These properties include its type or subtype (e.g. remote procedure call, pipeline, broadcast, shared data representation, document exchange standard, event), rules about the types of interfaces it works with, assurances about the interaction, commitments about the interaction such as ordering or performance, and so on. The specific named entities visible in a connector’s protocol are roles to be satisfied. The interface includes rules about the players that can match each role, together with other interaction properties. Components may be either primitive or composite. Primitive components may be implemented as code in a conventional programming language, shell scripts of the operating system, software developed in an application such as a spreadsheet, or other means external to the architectural description language. Composite components define configurations in a notation independent of conventional programming languages. This notation must be able to identify the constituent components and connectors, match the players of components with roles of connectors, and check that the resulting compositions satisfy the specifications of both the components’ interfaces and the connectors’ protocols. Similarly, connectors may be either primitive or composite. They are of many kinds: shared data representations, remote procedure calls, data flow, document exchange standards, standardized network protocols, etc. The connectors derived directly from programming languages are typically asymmetric with two complementary roles: a procedure has a definer and multiple callers; data has an
Mary Shaw
Abstractions for Software Architecture and Tools to Support Them
8
owner and multiple users; a simple data stream has one producer and one consumer. However, other, more abstract connectors may be symmetric, and their roles may be more numerous and more specific: in broadcast communication, all participants may be alike; in some event systems any component may raise or respond to any event. The usual import/export or provides/requires relation is too restrictive to express the relations used in practice—it fails to expose important distinctions. If rich, abstract connectors are to be defined, the architectural notation must be able to express complex interaction properties. These might be as diverse as ? ? ? ? ? ? guarantees about delivery of packets in a communication system; ordering restrictions on events using traces or path expressions; incremental production/consumption rules about pipelines; the distinction between the roles of clients and servers; parameter matching and binding rules for conventional procedure calls; restrictions on parameter types that can be used for remote procedure calls; and so on.
Primitive connectors may be implemented in a number of ways: as built-in mechanisms of programming languages (e.g., procedure calls or shared data); as system functions of the operating system (e.g., certain kinds of message passing); as library code in conventional programming languages (e.g., X/Motif); as shared data (e.g., Fortran COMMON or Jovial COMPOOL); as entries in task or routing tables; as a combination of library procedures and a single independent process for the connector (e.g., certain kinds of communication services); as interchange formats for static data (e.g., RTF); as initialization parameters (e.g., event period and process priority in a real-time operating system) and probably in a variety of other ways. Composite connectors may also appear in these diverse forms; we need (but do not yet have) ways to define them as well. The example of Figure 2, heterogeneous implementation of a pipeline, uses different types of components and connectors. It also shows a composite implementation of one of the filters. The remainder of this section deals with three issues of particular interest for general-purpose architectural tools: abstraction and encapsulation (Section 2.2), the appropriate analog for types (Section 2.3), and the ability to provide access to externally-developed tools (Section 2.4).
2.2.
Abstraction and Encapsulation
For a composite element, the implementation part consists of ? a parts list (components and connectors) ? composition instructions (association between roles and players) ? abstraction mapping (relation between internal players and players of the composite) ? other related specifications (detailed properties of the parts and compositions). This localizes and encapsulates information about the system structure rather than distributing it around the system in import/export statements. Since composition information is localized, global properties such as restrictions on topology or types of elements may be checked. Since the abstraction mapping is explicit, we have the opportunity to allocate responsibility for the code correctly implementing higher-level connectors. Further, the composition instructions make the matching of
Mary Shaw
Abstractions for Software Architecture and Tools to Support Them
9
players and roles explicit, so we can break free of name matching as the sole means of making connections, as shown in Section 3.3.3, Program 4. Common system-composition idioms, such as pipeline, client-server, or blackboard, can be defined as idiomatic patterns, or styles, of components and connectors. These patterns describe the types of components and connectors that can be used and may constrain the interconnection topologies. Indeed, some such styles (e.g., pipe-and-filter) are described primarily in terms of the prescribed form for communication, data sharing, or other interaction. In practice, the rules for these styles are usually implicit. The combination of localized definitions and higher-level elements makes it possible to formalize rules for styles [Abowd et al 93, Allen & Garlan 94a]. Abstract data types rely on an abstraction function to show the correspondence between the internal representation of a type and the abstract view that the user (and the specification) takes [Hoare 72]. For software architectures, abstractions are required to implement higher-level components in terms of lower-level ones. When a component or connector is not directly implemented by a programming language, its definition must explain how the abstract properties will be implemented. This might take the form of a manual or informal guidance. Even better, it could be a code template, an automated generator, or a formalization. No matter how it is represented, the definition must set out the programmer’s responsibility. When a number of components are connected to form a larger component, the players of the defined unit may be more abstract than the players of the implementation. In that case, the definition must explicitly indicate the correspondence among one or more external player (abstraction), one or more players of the constituent components, and the implementation rule. This is the abstraction mapping of the component. Similarly, abstraction mappings will be required for composite connectors. This differs from data abstraction chiefly in that it maps not only data to an abstract value but rather data plus functions to a set of abstract player types.
2.3.
Types and Type Checking
A problem similar to type checking in a programming language arises at four points in an architectural language. Two of these appear in the preceding discussion: the types of components and of connectors and their use in showing adherence to a style. As with any type system, types for components and connectors express the designer’s intent about how to use the element properly and are most useful when the language checks them. The types for connectors and components are not merely enumerations of unrelated items; some are closely related to others. Architectural types describe expected capabilities and limit both what can appear in the construct’s specification and the legitimate ways to use the construct. Examination of real systems shows that type hierarchies of this sort are useful. For example, there are many kinds of memories (components) and many kinds of event systems (connectors). Defining type structures for these elements requires the creation of taxonomies to catalog and structure the type variations. This is part of establishing a full model for architectural composition. The third place where type checking appears is at the actual point of associating a component’s players with a connector’s roles. Each of the named entities in the interfaces and protocols must have enough type information and other specification to determine whether the connector definition allows the components to be associated as requested. Furthermore, a component may be used differently by different kinds of connectors. For example, a client might be indifferent to whether its servers are dedicated, shared, or distributed. An abstract pipe may be able to connect both filters and files (but not processes that share data directly). We must therefore support flexible associations
Mary Shaw
Abstractions for Software Architecture and Tools to Support Them
10
between players and roles. For connectors such as procedure call that correspond directly to language constructs, this corresponds to the checks a linker may make. However, for richer, more abstract connectors the checks are more sophisticated. The fourth opportunity for type-like checking arises when more than one architectural style is used in designing a single system. This problem also resembles that of reconciling multiple views. A comparison of architectures for a single problem [Shaw 95] explores this issue. Components and connectors must be reusable in different settings, so it’s important to deal reasonably with associations that are slightly mismatched. Common examples include mismatches between the order and types of a library procedure’s arguments and those of the procedure intended to call it; remapping data formats to support sharing; and the subtle differences between remote procedure calls and local procedure calls. If component’s packaging fails to match the packaging needed in use, mechanical adaptation may be possible. Our current implementation makes initial steps toward supporting adaptation in the face of mismatch.
2.4.
Accommodating Analysis Tools
Architectural descriptions should be “open” with respect to analysis tools. We must accommodate techniques that are applied at the systems level of design. These analysis tools will often be developed independent of the model. They may address such properties as functional correctness, performance, and timing (e.g., allowable order of operations, real-time guarantees). The architectural description language should be able to interact with any analysis technique that works with information in the architectural specifications. It should be able to record the system-level specifications required by external tools as uninterpreted expressions, deliver information to the tools, receive results from the tools, and incorporate those results in the architectural description. Perhaps the most natural kind of system-level analysis is that of functional specification, which might use pre- and post-conditions to check procedure calls, for example. Perry suggested this as part of his software interconnection model [Perry 87]. Rather than building a theorem prover into the system, the system could collect the assertions from a procedure’s definer and a potential caller and invoke an external theorem prover to decide whether to allow the call. This will provide a much stronger check than either name matching or signature comparison. A second example is rigorous analysis of the real-time properties of a system. For certain classes of systems, correctness depends on the time at which computations are completed, not just on whether the computations themselves are correct. The designer of a system must account for the computation times of individual modules as well as the complex interactions within the composition of modules. As described in Section 3.5, our implementation now supports rate monotonic analysis (RMA), which is a body of techniques for analyzing the schedulability of preemptive, fixed-priority systems. The Software Engineering Institute at Carnegie Mellon has developed a handbook for this family of analysis techniques [Klein et al 93].
3.
UniCon:
Language for Universal Connector Support
In order to gain experience with the practical details of an architectural description language, we have implemented a simplified initial system. The system, called UniCon, emphasizes the structural aspects of software architecture. It is higher-level and more general than existing mechanisms for system composition, but it is low-level compared to the model of Section 2. Specifically, the objectives of this implementation were the following:
Mary Shaw Abstractions for Software Architecture and Tools to Support Them 11
? Address real problems of system description and composition; provide a prototype of a practical tool. ? Provide uniform access to a wide range of connection mechanisms. Select connectors that are available in the local computing environment so we can concentrate on providing them, not inventing them; however, select with diversity in mind. ? Help software designers to discriminate among different types of components and different types of connectors and to check the legitimacy of proposed configurations. ? Support both graphical and textual notations with interchange between the representations. ? Support analysis tools and specification notations developed by others. Preserve compatibility with programming tools in common use. ? Accept existing components. Use components written in ordinary programming languages, including those not written for use with this tool. ? Keep added run-time costs to a minimum. Ideally all UniCon-specific elements disappear by the time the executing system is initialized. To speed the process of gaining experience, some simplifications were made to the model when implementing UniCon, namely: ? UniCon supports composite components but not composite connectors. ? Abstractions for components and connectors are built in, but new ones cannot yet be defined. These built-in elements sample a diverse space, but the set is in no sense complete and a unifying taxonomy is not yet provided. ? The only primitive components at present are compilation units. Although the implementation is largely language-indifferent, C is the language supported at present. ? The syntax has not been refined for conciseness yet. It can be a bit wordy, especially when making intermodule connections of procedures and data with no change of name. This section begins by presenting the semantics of UniCon (Section 3.1). As an aid to intuition, this section anticipates the syntax discussion by providing a sample of the textual form of the KWIC example from Figure 1. The following two sections describe graphical syntax (Section 3.2) and textual syntax (Section 3.3); these are equivalent. We then describe the way existing code is incorporated as primitive components (Section 3.4) and the technique for using external analysis tools (Section 3.5).
3.1.
Semantics
Following the model presented in Section 2, UniCon is based on two complementary kinds of constructs: the component and the connector. Their structures are symmetric, except that composite connectors are not yet supported. Each has: ? a Name ? a specification (called an Interface for a component, a Protocol for a connector) ? a component or connector Type ? a set of global assertions in the form of a Property List ? a collection of association units: Players for components, Roles for connectors. Specific details about these are specified in Property Lists. ? an Implementation
Mary Shaw Abstractions for Software Architecture and Tools to Support Them 12
Each of these definitions can be understood solely in terms of its specification, and the properties of a system should be derivable from the specification of its components and connectors. This is closely related to Lam and Shankar’s properties separability and composability [Lam & Shankar 94]. Each provides a template that is instantiated when it is used. Information about distinctions among components, connectors, players, and roles is used for an analog of type checking. The definition of language semantics is organized around this structure. Section 3.1.1 describes components, Section 3.1.2 describes connectors, and Section 3.1.3 describes type checking.
3 . 1 . 1 . Components
Components define computational capabilities. A component consists of an interface that specifies the capabilities the component exports and an implementation, which may be primitive or composite. The interface of a component must be consistent with its implementation. In the case of a primitive implementation, such as a source file or executable, this is the responsibility of the programmer. In the case of a composite implementation, the interface must be consistent with the interfaces of its member components and the protocols of its connectors; its semantics should be derivable from the interfaces of its constituents. The interface defines the computational commitments the component can make and constraints on the way the component is to be used. The interface provides the guarantees that will hold of the behavior and performance of the component. It should be possible to use the component by reference to the interface alone. The interface must include ? the component type ? assertions and constraints that apply to the entire component ? the players defined by the component, which each consist of a name and type and optional attributes like signature, functional specifications, constraints on use, or information required specifically by a component type (e.g., port bindings of Unix file descriptors). A component type expresses the designer's intention about the general class of functionality to be provided by the component; it restricts the numbers, types, and specifications of the Players defined by the component. The players, which form the bulk of the interface, are the visible semantic units through which the component can interact, request and provide services, or be influenced by external state or events. The interactions are mediated by the roles of connectors. The detailed specifications of the players appear in the form of property lists, which are lists of attributes and their associated values. Program 1 shows an example of the textual description of both a primitive and a composite component, which will be referenced throughout this section. Component types and their associated players are discussed below in Section 3.1.1.1. To provide some specific intuition for the semantics, we will use the pipeline implementation of KWIC presented in Figure 1. This example’s pipeline involves four filters, one file, four pipes, and the two streams that are Players in the composed system. Three of the filters in this example have the canonical interface for a simple filter, specifying stdin, stdout, and stderr; merge, of course, has two inputs. For these filters the specification further indicates that the data on the stream is line-oriented. Program 1 shows the textual notation for two components from the KWIC example, which will be referenced throughout this section. Component KWIC from Program 1 is an example of a system. A system is a component, usually a composite component, that is capable of independent operation in the context of a computer and its operating and runtime systems. The external specification of the system is the interface of the com-
Mary Shaw
Abstractions for Software Architecture and Tools to Support Them
13
ponent. The system remains eligible for use as a component; indeed, the essence of systems integration is finding ways to treat as subsystems today those things that were systems last week. A system may interact with users and other systems. A system is closed in the sense that all the players of its interface are bound to its execution environment; it implements a function that is discrete and complete in the mind of its designer. This is the result of an architectural definition.
COMPONENT sort INTERFACE IS TYPE Filter PLAYER input IS StreamIn SIGNATURE ("line") PORTBINDING (stdin) END input PLAYER output IS StreamOut SIGNATURE ("line") PORTBINDING (stdout) END output PLAYER error IS StreamOut SIGNATURE ("line") PORTBINDING (stderr) END error END INTERFACE IMPLEMENTATION IS VARIANT sort IN "sort" IMPLTYPE (Executable) INITACTUALS ("-f") END sort END IMPLEMENTATION END sort COMPONENT KWIC INTERFACE IS TYPE Filter PLAYER input IS StreamIn SIGNATURE ("line") PORTBINDING (stdin) END input PLAYER output IS StreamOut SIGNATURE ("line") PORTBINDING (stdout) END output PLAYER error IS StreamOut SIGNATURE ("line") PORTBINDING (stderr) END error END INTERFACE IMPLEMENTATION IS /* First instantiate the parts to use */ USES caps INTERFACE upcase USES shifter INTERFACE cshift USES req-data INTERFACE const-data USES merge INTERFACE converge USES sorter INTERFACE sort USES P PROTOCOL Unix-pipe USES Q PROTOCOL Unix-pipe USES R PROTOCOL Unix-pipe /* Associate players of some parts to players of interface. By default, all stderrs are bound to the external stderr */ BIND input TO caps.input BIND output TO sorter.output /* Describe the way KWIC is built from parts by directly associating players with roles */ CONNECT caps.output TO P.source CONNECT shifter.input TO P.sink CONNECT shifter.output TO Q.source CONNECT req-data.read TO R.source CONNECT merge.in1 TO R.sink CONNECT merge.in2 TO Q.sink /* Syntactic sugar for full connections */ ESTABLISH Unix-pipe WITH merge.output AS source sorter.input AS sink END Unix-pipe END IMPLEMENTATION END KWIC
Program 1. A primitive and a composite component.
A component’s implementation may be either primitive or composite. Primitive implementations provide one or more ways to locate a definition in some programming language and are discussed
Mary Shaw Abstractions for Software Architecture and Tools to Support Them 14
below in Section 3.1.1.2. Composite implementations instantiate a set of components and configure them with connectors and are discussed in Section 3.1.1.3.
ComponentType Module Computation SharedData SeqFile Filter Process SchedProcess General Player Types supported RoutineDef, RoutineCall, GlobalDataDef, GlobalDataUse, PLBundle, ReadFile, WriteFile RoutineDef, RoutineCall, GlobalDataUse, PLBundle GlobalDataDef, GlobalDataUse, PLBundle ReadNext, WriteNext StreamIn, StreamOut RPCDef, RPCCall RPCDef, RPCCall, RTLoad All (that is, any player type is allowed)
Table 1. Built-in component types and their players. 3.1.1.1. Built-in Component Types
Component types and player types are currently defined by enumeration, but extensions may be supported some day. Table 1 lists the component types currently supported and the players allowed for each. This set of component types was selected opportunistically: we wanted to reflect components and connectors used in practice and to cover as wide a variety as possible. Each of these is described in detail below. ? Component type Module corresponds to a compilation unit in a typical programming language. The RoutineDef players correspond to exported procedures and functions. The RoutineCall players correspond to imported procedures and functions. Similarly, GlobalDataDef and GlobalDataUse players correspond to import and export of named data. The ReadFile and WriteFile players provide an input/output capability.3 These players correspond directly to language constructs or system calls. However, modules frequently define one or more coherent collections of players; when designers think about the architecture of the system, they think about the use of a collection of players rather than about all the individuals. UniCon captures this with an abstract player, PLBundle, that corresponds to a set of individual players related to procedure calls or data use. Modules are intended to provide definitions that will be linked in a single name space. ? Component type Computation is a specialization of Module whose interface is restricted to defining procedures, functions, and bundles thereof and using procedures, functions, and data defined elsewhere. It is intended to capture purely computational units that are collections of procedure definitions and calls. Similarly, component type SharedData is a specialization of Module whose interface is restricted to defining and referencing data. ? Component type SeqFile corresponds to sequential files in which lines, characters, or records (as specified by the RecordFormat attribute) are read sequentially from the front (ReadNext) and written sequentially at the end (WriteNext).
3This
currently makes no provision for interactive input and output. This will be addressed soon by adding a connector for typescripts, later by adding a connector that provides protocol for interaction with a user-defined graphical interface.
Mary Shaw Abstractions for Software Architecture and Tools to Support Them 15
? Component type Filter corresponds to Unix filters in which input arrives in streams and output is produced in streams (StreamIn and StreamOut). The syntax of the stream (the structure imposed on elements in the stream) may be specified by the Signature attribute. ? Component type Process corresponds to an independently scheduled process at the operating system level. It differs from Filter in that it interacts via remote procedure calls (RPCDef and RPCCall) rather than data flow.4 SchedProcess is a special type of Process that admits of real-time analysis and scheduling; (see Section 3.5). A SchedProcess provides an abstract player RTLoad that provides the information required for real-time scheduling using attributes SegmentSet, Trigger, SegmentDef, and TriggerDef. ? A General component type allows any player types, thus allowing the definition of arbitrary components; it does not support analysis or checking. Using general components when it is possible to use more specific types defeats the purpose of component typing.
4Interprocess
data sharing will be added in the future.
Abstractions for Software Architecture and Tools to Support Them 16
Mary Shaw
Attribute InstFormals
Req/Opt Merge Applies to Rule Components optional merge all
Value / Default / Syntax Formal parameter list for instantiation of component. Default is no instantiation parameters. Syntax depends on rules of implementation language. Used during instantiation of the component to select among variant implementations. Default is the first implementation provided. Syntax is name of a variant. Format of records in the file. Default is lines separated by
Variant
optional replace all
RecordFormat optional replace SeqFile Library optional replace Computation, Module, SeqFile, SharedData optional replace Module, Computation, Process, SchedProc optional replace SchedProc optional replace All required replace SchedProc
EntryPoint
Priority Processor SegmentDef
Priority at which real-time operating system should run the process. Default is highest. Syntax is integer. Processor on which the component will be executable. Default is local processor. Syntax is processor name. Definition of a segment of code in the implementation of a real-time component. Syntax is name followed by ‘;’ followed by execution time in seconds. No default Definition of external stimulus that activates a segment. Syntax is name followed by ‘;’ followed by period of stimulus in seconds. No default. Auxiliary information required to derive language-independent type information for RPC generator. Auxiliary information required to derive language-independent type information for RPC generator.
TriggerDef
optional replace SchedProc
RPCTypesIn RPCTypedef
optional merge Process, SchedProc optional replace Process, SchedProc
Table 2. Attributes that apply to components.
Attributes in property lists provide further specification of a component. Each attribute is relevant to one or more component types and may be required or optional; it must specify what to do if multiple values are specified for the attribute, for example, in different property lists. The possibilities are currently to use the most recent (replace), to construct a list of all values (merge), and to refuse multiple definitions (error). Certain attributes pertain to a component as a whole. For example, in cases where it is possible to provide instantiation parameters to an entire component, those are provided with the property list for the component itself. Table 2 lists component attributes, and Table 3 gives the attributes defined for Players. They are largely specific to the type of the Player.
Mary Shaw
Abstractions for Software Architecture and Tools to Support Them
17
Attribute MaxAssocs MinAssocs Signature
Req/Opt Merge Applies to Rule Players optional replace all optional replace all required error RoutineDef, RoutineCall, GlobalDataDef, GlobalDataUse, StreamIn, StreamOut, ReadFile, WriteFile, RPCDef, RPCCall
Value / Default / Syntax Maximum number of associations the player can be involved in. Default is unlimited. Syntax is integer. Minimum number of associations the player can be involved in. Default is 1. Syntax is integer. Formal parameter list or type definition, possibly with default values for the parameters. No default value; this is such an essential part of the definition of these players that even an empty parameter list should be explicitly indicated. Syntax is dependent on type of player and implementation language.
Portbinding
required replace StreamIn, StreamOut required replace PLBundle
Binding of port for pipe. Defaults are stdin for StreamIn and stdout for StreamOut. Syntax is integer or port id such as stdin, stdout, stderr. Defines a player to be a member of an abstract collection of players that correspond to programming language constructs procedure and data. Syntax is name, player type, and property list, separated by ‘;’. No default Specifies the set of segments that make up a RTLoad. Syntax is a set of SegmentDefs. No default. Specifies a trigger to be used in a RTLoad. Syntax is TriggerDef. no default.
Member
SegmentSet Trigger
required replace RTLoad optional replace RTLoad
Table 3. Attributes that apply to Players. 3.1.1.2. Implementation of Primitive Components
Primitive components are implemented directly in the code of some programming language or data stored in files. They are made available to UniCon by providing an appropriate specification as a wrapper. Code may be represented as source, object, or executable; other representations will be added as required. Since multiple representations for a component may be available (for example, both source and object code), UniCon allows multiple representations, called variants, to be specified. When a primitive component is instantiated, the Variant attribute can be used to select the preferred variant. This capability can be exploited, for example, to provide variants with different performance properties or variants with special support for debugging or monitoring. Table 4 shows the attributes defined for primitive implementations of components. The KWIC indexer example is constructed from primitive filter components, like the Sort component shown in Program 1. The interfaces of the component filters are very similar to the interface for the whole system. The implementations are primitive, and since they correspond directly to executable Unix filters, they have only one variant.
Mary Shaw
Abstractions for Software Architecture and Tools to Support Them
18
Attribute ImplType
Req/Opt Merge Applies to Rule Components optional error all
Value / Default / Syntax Representation: source, object, executable, data, or whatever. Default is to refer to file extension. Syntax is literal from enumeration of known kinds Actual parameters for initialization Options passed to the system builder. Syntax is option syntax for system builder. No default.
InitActuals BuildOption
optional error
all
optional merge all
Table 4. Attributes defined for primitive components. 3.1.1.3. Implementation of Composite Components
Composite components provide the capability of building up progressively larger subsystems from components of (potentially different) types. A composite implementation must provide three kinds of information: ? The parts: Instantiations of the components and connectors from which the composite component is constructed. ? The configuration: Specification of how the instantiations of connectors link the instantiations of components, i.e. the associations5 between players and roles. ? The abstraction: Specification of how the players of the interface will be associated with players of the implementation. Since more than one instance of an element (component or connector) definition may be used in a single implementation, these definitions are templates to be instantiated for each use. Each instantiation may provide a property list that further constrains the attributes of the element. The merge rule of the attribute determines the interpretation when multiple values are provided for an attribute. The configuration of the composite implementation is defined by explicitly connecting players and roles. Each match is checked by comparing the type of the player against the Accepts attribute of the component, ensuring that maximum and minimum connection counts are satisfied. The interface of the component being defined specifies the players that the component must provide; these are the ExternalPlayers. In the implementation, players are provided when components are instantiated; these are the InternalPlayers. The abstraction step defines ExternalPlayers in terms of InternalPlayers. Two cases arise: ? In the simple case, one of the constituent components defines an InternalPlayer of the same type and specification as the ExternalPlayer. In this case, simple name binding suffices to export the player through the interface. ? The more complex case arises when an ExternalPlayer is of a type not directly supported by the programming language. In this case the ExternalPlayer must be implemented by more concrete InternalPlayers of the implementation. This is the case, for example, when a StreamIn is implemented with calls on library routines that read standard input. It will become increasingly common as we add more abstract players that are realized as calls on several specific procedures according to set protocols. The definition of such an abstract
5An
association is either a bind (to an external player of a component) or a connect (to a role of a connector).
Mary Shaw Abstractions for Software Architecture and Tools to Support Them 19
connector must specify the functionality that the implementation must provide. In this case, the binding must indicate the name of the ExternalPlayer in the interface and show how InternalPlayers satisfy the required functionality. The special attribute MapsTo identifies the InternalPlayer(s). Warnings about ExternalPlayers and roles that are not associated with InternalPlayers are given as appropriate. Setting the MinAssocs attribute of a Player to 0 indicates that it is normal for the Player to be unassociated. A Library attribute will soon be provided to indicate that many players will normally remain unassociated. In the implementation of KWIC in Program 1, four filters and a file are used to build a system that is itself a filter. The definition of pipe used here corresponds to unix: as indicated in Table 5, it supports the players of sequential files as well as filters. The implementation is composite and has three parts. First, it instantiates the parts to be used. Next, it binds the StreamIn of the first filter (caps) to the StreamIn of the interface and the StreamOut of the last filter (sorter) to the StreamOut of the interface. (This is an example of the simple case where ExternalPlayers are bound to InternalPlayers of the same type.) Finally, it configures the system by indicating which inputs and outputs are connected by which pipes. The example shows two ways to do this: by individually associating players to roles of explicitly instantiated pipe connectors and by a single statement that implicitly instantiates the pipe and makes all connections at once. The former allows the connections to be grouped by the designer’s choice; the latter assures the reader that no other roles of the connector are connected elsewhere. The mechanical character of the example provides ample motivation for using the graphical interface described in Section 3.2.
3 . 1 . 2 . Connectors
Connectors mediate interactions among components. A connector consists of a protocol that specifies the class of interactions the connector provides and an implementation. Connectors define the protocols and mechanics of interaction together with any additional mechanisms required to carry out the interaction: auxiliary data structures, initialization routines, and so on. The connector definition is also the location for specifications of required behavior such as interchange representations and the internal manifestation (e.g. sequence of procedure calls) of the connector in the code of a component. At present, all connectors are primitive and their implementations are therefore individually crafted within the UniCon implementation. The protocol defines the allowable interactions among a collection of components and provides guarantees about those interactions. To do this it defines roles, or the responsibilities of various parties that set requirements for the players of components whose interactions are to be governed by the connector. The author of the component is responsible for ensuring that these responsibilities are satisfied by the implementation. The protocol must include: ? the connector type ? assertions that constrain the entire connector (for example, rules about timing or ordering); these are the commitments about the interaction that the protocol supports ? the roles that participate in the protocol; each consists of a name and type and optional attributes like signature, functional specifications, or constraints on their use. A connector type expresses the designer's intention about the general class of connection to be provided by the connector; it restricts the numbers, types, and specifications of the Roles provided by the connector. In particular, some roles may require players, some may be optional but constrained
Mary Shaw
Abstractions for Software Architecture and Tools to Support Them
20
大数据分析的六大工具介绍 2016年12月 一、概述 来自传感器、购买交易记录、网络日志等的大量数据，通常是万亿或EB的大小，如此庞大的数据，寻找一个合适处理工具非常必要，今天我们为大家分学在大数据处理分析过程中六大最好用的工具。 我们的数据来自各个方面，在面对庞大而复杂的大数据，选择一个合适的处理工具显得很有必要，工欲善其事，必须利其器，一个好的工具不仅可以使我们的工作事半功倍，也可以让我们在竞争日益激烈的云计算时代，挖掘大数据价值，及时调整战略方向。 大数据是一个含义广泛的术语，是指数据集，如此庞大而复杂的，他们需要专门设il?的硬件和软件工具进行处理。该数据集通常是万亿或EB的大小。这些数据集收集自各种各样的来源:传感器、气候信息、公开的信息、如杂志、报纸、文章。大数据产生的其他例子包括购买交易记录、网络日志、病历、事监控、视频和图像档案、及大型电子商务。大数据分析是在研究大量的数据的过程中寻找模式, 相关性和其他有用的信息，可以帮助企业更好地适应变化，并做出更明智的决策。 二.第一种工具:Hadoop Hadoop是一个能够对大量数据进行分布式处理的软件框架。但是Hadoop是 以一种可黑、高效、可伸缩的方式进行处理的。Hadoop是可靠的，因为它假设计算元素和存储会失败，因此它维护多个工作数据副本，确保能够针对失败的节点重新分布处理。Hadoop 是高效的，因为它以并行的方式工作，通过并行处理加快处理速度。Hadoop还是可伸缩的，能够处理PB级数据。此外，Hadoop依赖于社区服务器，因此它的成本比较低，任何人都可以使用。
Hadoop是一个能够让用户轻松架构和使用的分布式计算平台。用户可以轻松地 在Hadoop上开发和运行处理海量数据的应用程序。它主要有以下儿个优点: ,高可黑性。Hadoop按位存储和处理数据的能力值得人们信赖。，高扩展性。Hadoop是 在可用的计?算机集簇间分配数据并完成讣算任务 的，这些集簇可以方便地扩展到数以千计的节点中。 ，高效性。Hadoop能够在节点之间动态地移动数据，并保证各个节点的动 态平衡，因此处理速度非常快。 ，高容错性。Hadoop能够自动保存数据的多个副本，并且能够自动将失败 的任务重新分配。 ,Hadoop带有用Java语言编写的框架，因此运行在Linux生产平台上是非 常理想的。Hadoop上的应用程序也可以使用其他语言编写，比如C++。 第二种工具:HPCC HPCC, High Performance Computing and Communications（高性能计?算与通信）的缩写° 1993年，山美国科学、工程、技术联邦协调理事会向国会提交了“重大挑战项 U：高性能计算与通信”的报告，也就是被称为HPCC计划的报告，即美国总统科学战略项U ,其U的是通过加强研究与开发解决一批重要的科学与技术挑战 问题。HPCC是美国实施信息高速公路而上实施的计?划，该计划的实施将耗资百亿 美元，其主要U标要达到:开发可扩展的计算系统及相关软件，以支持太位级网络 传输性能，开发千兆比特网络技术，扩展研究和教育机构及网络连接能力。
一、基因芯片： 1、基因芯片综合分析软件。 ArrayVision 7.0 一种功能强大的商业版基因芯片分析软件，不仅可以进行图像分析，还可以进行数据处理，方便protocol的管理功能强大，商业版正式版：6900美元。 Arraypro 4.0 Media Cybernetics公司的产品，该公司的gelpro, imagepro一直以精确成为同类产品中的佼佼者，相信arraypro也不会差。 phoretix™ Array Nonlinear Dynamics公司的基因片综 合分析软件。 J-express 挪威Bergen大学编写，是一个用JAVA语言写的应用程序，界面清晰漂亮，用来分析微矩阵（microarray）实验获得的基因表达数据，需要下载安装JAVA运行环境JRE1.2后(5.1M)后，才能运行。 2、基因芯片阅读图像分析软件 ScanAlyze 2.44 ，斯坦福的基因芯片基因芯片阅读软件，进行微矩阵荧光图像分析，包括半自动定义格栅与像素点分析。输出为分隔的文本格式，可很容易地转化为任何数据库。
3、基因芯片数据分析软件 Cluster 斯坦福的对大量微矩阵数据组进行各种簇（Cluster)分析与其它各种处理的软件。 SAM Significance Analysis of Microarrays 的缩写，微矩阵显著性分析软件，EXCEL软件的插件，由Stanford大学编制。4．基因芯片聚类图形显示 TreeView 1.5 斯坦福开发的用来显示Cluster软件分析的图形化结果。现已和Cluster成为了基因芯片处理的标准软件。 FreeView 是基于JAVA语言的系统树生成软件，接收Cluster生成的数据，比Treeview增强了某些功能。 5．基因芯片引物设计 Array Designer 2.00 DNA微矩阵（microarray）软件，批量设计DNA和寡核苷酸引物工具 二、RNA二级结构。 RNA Structure 3.5 RNA Sturcture 根据最小自由能原理，将Zuker的根据RNA
以下是我在近三年做各类计量和统计分析过程中感受最深的东西，或能对大家有所帮助。当然，它不是ABC的教程，也不是细致的数据分析方法介绍，它只是“总结”和“体会”。由于我所学所做均甚杂，我也不是学统计、数学出身的，故本文没有主线，只有碎片，且文中内容仅为个人观点，许多论断没有数学证明，望统计、计量大牛轻拍。 于我个人而言，所用的数据分析软件包括EXCEL、SPSS、STATA、EVIEWS。在分析前期可以使用EXCEL进行数据清洗、数据结构调整、复杂的新变量计算（包括逻辑计算）；在后期呈现美观的图表时，它的制图制表功能更是无可取代的利器；但需要说明的是，EXCEL毕竟只是办公软件，它的作用大多局限在对数据本身进行的操作，而非复杂的统计和计量分析，而且，当样本量达到“万”以上级别时，EXCEL的运行速度有时会让人抓狂。 SPSS是擅长于处理截面数据的傻瓜统计软件。首先，它是专业的统计软件，对“万”甚至“十万”样本量级别的数据集都能应付自如；其次，它是统计软件而非专业的计量软件，因此它的强项在于数据清洗、描述统计、假设检验（T、F、卡方、方差齐性、正态性、信效度等检验）、多元统计分析（因子、聚类、判别、偏相关等）和一些常用的计量分析（初、中级计量教科书里提到的计量分析基本都能实现），对于复杂的、前沿的计量分析无能为力；第三，SPSS主要用于分析截面数据，在时序和面板数据处理方面功能了了；最后，SPSS兼容菜单化和编程化操作，是名副其实的傻瓜软件。 STATA与EVIEWS都是我偏好的计量软件。前者完全编程化操作，后者兼容菜单化和编程化操作；虽然两款软件都能做简单的描述统计，但是较之 SPSS差了许多；STATA与EVIEWS都是计量软件，高级的计量分析能够在这两个软件里得到实现；STATA的扩展性较好，我们可以上网找自己需要的命令文件（.ado文件），不断扩展其应用，但EVIEWS 就只能等着软件升级了；另外，对于时序数据的处理，EVIEWS较强。 综上，各款软件有自己的强项和弱项，用什么软件取决于数据本身的属性及分析方法。EXCEL适用于处理小样本数据，SPSS、 STATA、EVIEWS可以处理较大的样本；EXCEL、SPSS适合做数据清洗、新变量计算等分析前准备性工作，而STATA、EVIEWS在这方面较差；制图制表用EXCEL；对截面数据进行统计分析用SPSS，简单的计量分析SPSS、STATA、EVIEWS可以实现，高级的计量分析用 STATA、EVIEWS，时序分析用EVIEWS。 关于因果性 做统计或计量，我认为最难也最头疼的就是进行因果性判断。假如你有A、B两个变量的数据，你怎么知道哪个变量是因（自变量），哪个变量是果（因变量）？ 早期，人们通过观察原因和结果之间的表面联系进行因果推论，比如恒常会合、时间顺序。但是，人们渐渐认识到多次的共同出现和共同缺失可能是因果关系，也可能是由共同的原因或其他因素造成的。从归纳法的角度来说，如果在有A的情形下出现B，没有A的情形下就没有B，那么A很可能是B的原因，但也可能是其他未能预料到的因素在起作用，所以，在进行因果判断时应对大量的事例进行比较，以便提高判断的可靠性。 有两种解决因果问题的方案：统计的解决方案和科学的解决方案。统计的解决方案主要指运用统计和计量回归的方法对微观数据进行分析，比较受干预样本与未接受干预样本在效果指标（因变量）上的差异。需要强调的是，利用截面数据进行统计分析，不论是进行均值比较、频数分析，还是方差分析、相关分析，其结果只是干预与影响效果之间因果关系成立的必要条件而非充分条件。类似的，利用截面数据进行计量回归，所能得到的最多也只是变量间的数量关系；计量模型中哪个变量为因变量哪个变量为自变量，完全出于分析者根据其他考虑进行的预设，与计量分析结果没有关系。总之，回归并不意味着因果关系的成立，因果关系的判定或推断必须依据经过实践检验的相关理论。虽然利用截面数据进行因果判断显得勉强，但如果研究者掌握了时间序列数据，因果判断仍有可为，其
手机微信QQ聊天记录取证方法（非常详细实用） 如今，由于互联网为交易双方提供了跨越物理空间的沟通渠道，通过网络协商订立合同条款越来越成为追求交易便捷、降低成本的商事主体的选择。 在案件当中，通过微信、QQ、电子邮件协商确定双方权利义务的情况比比皆是，也成为认定案件关键事实的主要证据。目前互联网电子数据证据成为案件主要证据的比例逐步上升，甚至部分案件中此类证据，特别是微信证据成为当事人证明自己主张的唯一证据。 为了统一司法实践中互联网电子证据认定标准，南沙法院在全省率先出台《互联网电子数据证据举证、认证规程(试行)》(以下简称《规程》)。 《规程》对互联网电子数据证据的内涵予以了界定，依据及参考已经颁布施行的《电子签名法》以及尚在审议过程中的《电子商务法》，并结合法院审判实践中出现较多的证据类型，列举了短信、电子邮件、QQ、微信、支付宝或其他具备通讯、支付功能的互联网软件所产生的，能够有形地表现所载内容并可以随时调取查用的数据信息。 针对各种证据类型，《规程》对当事人向法院提供证据方面作出指引，包括固化证据的形式、使用的介质、展示证据的程序。同时法院也在立案窗口和
应诉送达时专门放置举证指引清单，对于提交此类证据的当事人及时进行指导，减少不规范的举证行为，提高审判效率。 《规程》还对法官采信电子证据的认证过程作出指引。对于通过公证、鉴证可以解决证据真实性认证问题的，引导当事人进行公证、鉴证，而对于没有进行公证、鉴证或者公证、鉴证内容尚不足以证实证据真实性的情况，主要通过电子数据证据中当事人的身份真实性、通讯内容的完整性来对证据进行认证。 ▌关于微信相关证据《规程》这样说 当事人提交微信相关证据需注意以下三方面： 1、使用终端设备登陆本方微信账户的过程演示。用于证明其持有微信聊天记录的合法性和本人身份的真实性。 2、聊天双方的个人信息界面。借助微信号不可更改的特点，并结合个人信息界面中显示的手机号码、头像等信息固定双方当事人的真实身份。 3、完整的聊天记录。根据微信聊天记录在使用终端中只能删除不能添加的特点，根据双方各自微信客户端完整聊天信息进行对比，以验证相关信息的完整性和真实性。 法官采信微信相关证据时如何做? 1、法官在采信微信相关证据时如何确定微信使用者身份? 微信并未强制实名认证，法官在采信微信相关证据时，根据当事人提供的对方微信号、绑定的手机号码以及聊天记录中透露的相关信息内容，法官可以
常用统计软件介绍
常用统计软件介绍 《概率论与数理统计》是一门实践性很强的课程。但是,目前在国内,大多侧重基本方法的介绍,而忽视了统计实验的教学。这样既不利于提高学生创新精神和实践能力,也使得这门课程的教学显得枯燥无味。为此,我们介绍一些常用的统计软件，以使学生对统计软件有初步的认识,为以后应用统计方法解决实际问题奠定初步的基础。 一、统计软件的种类 1.SAS 是目前国际上最为流行的一种大型统计分析系统，被誉为统计分析的标准软件。尽管价格不菲，SAS已被广泛应用于政府行政管理，科研，教育，生产和金融等不同领域，并且发挥着愈来愈重要的作用。目前SAS已在全球100多个国家和地区拥有29000多个客户群，直接用户超过300万人。在我国，国家信息中心，国家统计局，卫生部，中国科学院等都是SAS系统的大用户。尽管现在已经尽量“傻瓜化”，但是仍然需要一定的训练才可以使用。因此,该统计软件主要适合于统计工作者和科研工作者使用。 2.SPSS SPSS作为仅次于SAS的统计软件工具包，在社会科学领域有着广泛的应用。SPSS是世界上最早的统计分析软件，由美国斯坦福大学的三位研究生于20世纪60年代末研制。由于SPSS容易操作，输出漂亮，功能齐全，价格合理,所以很快地应用于自然科学、技术科学、社会科学的各个领域，世界上许多有影响的报刊杂志纷纷就SPSS 的自动统计绘图、数据的深入分析、使用方便、功能齐全等方面给予了高度的评价与称赞。迄今SPSS软件已有30余年的成长历史。全球
约有25万家产品用户，它们分布于通讯、医疗、银行、证券、保险、制造、商业、市场研究、科研教育等多个领域和行业，是世界上应用最广泛的专业统计软件。在国际学术界有条不成文的规定，即在国际学术交流中，凡是用SPSS软件完成的计算和统计分析，可以不必说明算法，由此可见其影响之大和信誉之高。因此,对于非统计工作者是很好的选择。 3.Excel 它严格说来并不是统计软件，但作为数据表格软件，必然有一定统计计算功能。而且凡是有Microsoft Office的计算机，基本上都装有Excel。但要注意，有时在装 Office时没有装数据分析的功能，那就必须装了才行。当然，画图功能是都具备的。对于简单分析，Excel 还算方便，但随着问题的深入，Excel就不那么“傻瓜”，需要使用函数，甚至根本没有相应的方法了。多数专门一些的统计推断问题还需要其他专门的统计软件来处理。 4.S-plus 这是统计学家喜爱的软件。不仅由于其功能齐全，而且由于其强大的编程功能，使得研究人员可以编制自己的程序来实现自己的理论和方法。它也在进行“傻瓜化”,以争取顾客。但仍然以编程方便为顾客所青睐。 5.Minitab 这个软件是很方便的功能强大而又齐全的软件，也已经“傻瓜化”，在我国用的不如SPSS与SAS那么普遍。
Chapter4 Introduction to Analysis-of-Variance Procedures Chapter T able of Contents 52Chapter4.Introduction to Analysis-of-Variance Procedures SAS OnlineDoc?:Version8 Chapter4 Introduction to Analysis-of-Variance Procedures 54Chapter4.Introduction to Analysis-of-Variance Procedures The following section presents an overview of some of the fundamental features of analysis of variance.Subsequent sections describe how this analysis is performed with procedures in SAS/STAT software.For more detail,see the chapters for the individual procedures.Additional sources are described in the“References”section on page61. De?nitions Analysis of variance(ANOV Ais a technique for analyzing experimental data in which one or more response(or dependent or simply Yvariables are measured un-der various conditions identi?ed by one or more classi?cation variables.The com-binations of levels for the classi?cation variables form the cells of the experimental design for the data.For example,an experiment may measure weight change(the dependent variablefor men and women who participated in three different weight-loss programs.The six cells of the design are formed by the six combinations of sex (men,womenand program(A,B,C.
大数据可视化分析平台 一、背景与目标 基于邳州市电子政务建设的基础支撑环境，以基础信息资源库(人口库、法人库、宏观经济、地理库)为基础，建设融合业务展示系统，提供综合信息查询展示、信息简报呈现、数据分析、数据开放等资源服务应用。实现市府领导及相关委办的融合数据资源视角，实现数据信息资源融合服务与创新服务，通过系统达到及时了解本市发展的综合情况，及时掌握发展动态,为政策拟定提供依据。 充分运用云计算、大数据等信息技术，建设融合分析平台、展示平台，整合现有数据资源,结合政务大数据的分析能力与业务编排展示能力，以人口、法人、地理，人口与地理,法人与地理，实现基础展示与分析,融合公安、交通、工业、教育、旅游等重点行业的数据综合分析，为城市管理、产业升级、民生保障提供有效支撑。 二、政务大数据平台 1、数据采集和交换需求:通过对各个委办局的指定业务数据进行汇聚,将分散的数据进行物理集中和整合管理，为实现对数据的分析提供数据支撑。将为跨机构的各类业务系统之间的业务协同，提供统一和集中的数据交互共享服务。包括数据交换、共享和EＴL等功能。 ２、海量数据存储管理需求：大数据平台从各个委办局的业务系统里抽取的数据量巨大,数据类型繁杂,数据需要持久化的存储和访问。不论是结构化数据、半结构化数据，还是非结构化数据,经过数据存储引擎进行建模后，持久化保存在存储系统上。存储系统要具备
高可靠性、快速查询能力。 3、数据计算分析需求：包括海量数据的离线计算能力、高效即席数据查询需求和低时延的实时计算能力。随着数据量的不断增加，需要数据平台具备线性扩展能力和强大的分析能力,支撑不断增长的数据量，满足未来政务各类业务工作的发展需要，确保业务系统的不间断且有效地工作。 4、数据关联集中需求：对集中存储在数据管理平台的数据，通过正确的技术手段将这些离散的数据进行数据关联,即:通过分析数据间的业务关系，建立关键数据之间的关联关系，将离散的数据串联起来形成能表达更多含义信息集合，以形成基础库、业务库、知识库等数据集。 5、应用开发需求:依靠集中数据集,快速开发创新应用，支撑实际分析业务需要。 6、大数据分析挖掘需求:通过对海量的政务业务大数据进行分析与挖掘,辅助政务决策,提供资源配置分析优化等辅助决策功能， 促进民生的发展。
常用分子生物学软件 一、基因芯片： 1、基因芯片综合分析软件。 ArrayVision 一种功能强大的商业版基因芯片分析软件，不仅可以进行图像分析，还可以进行数据处理，方便protocol的管理功能强大，商业版正式版：6900美元。 Arraypro Media Cybernetics公司的产品，该公司的gelpro, imagepro一直以精确成为同类产品中的佼佼者，相信arraypro也不会差。 phoretix? Array Nonlinear Dynamics公司的基因片综合分析软件。 J-express 挪威Bergen大学编写，是一个用JAVA语言写的应用程序，界面清晰漂亮，用来分析微矩阵（microarray）实验获得的基因表达数据，需要下载安装JAVA运行环境后后，才能运行。 2、基因芯片阅读图像分析软件 ScanAlyze ，斯坦福的基因芯片基因芯片阅读软件，进行微矩阵荧光图像分析，包括半自动定义格栅与像素点分析。输出为分隔的文本格式，可很容易地转化为任何数据库。 3、基因芯片数据分析软件 Cluster 斯坦福的对大量微矩阵数据组进行各种簇（Cluster)分析与其它各种处理的软件。 SAM Significance Analysis of Microarrays 的缩写，微矩阵显著性分析软件，EXCEL软件的插件，由Stanford大学编制。 4．基因芯片聚类图形显示 TreeView 斯坦福开发的用来显示Cluster软件分析的图形化结果。现已和Cluster成为了基因芯片处理的标准软件。 FreeView 是基于JAVA语言的系统树生成软件，接收Cluster生成的数据，比Treeview增强了某些功能。 5．基因芯片引物设计 Array Designer DNA微矩阵（microarray）软件，批量设计DNA和寡核苷酸引物工具 二、RNA二级结构。 RNA Structure RNA Sturcture 根据最小自由能原理，将Zuker的根据RNA一级序列预测RNA二级结构的算法在软件上实现。预测所用的热力学数据是最近由Turner实验室获得。提供了一些模块以扩展Zuker算法的能力，使之为一个界面友好的RNA折叠程序。允许你同时打开多个数据处理窗口。主窗口的工具条提供一些基本功能：打开文件、导入文件、关闭文件、设置程序参数、重排窗口、以及即时帮助和退出程序。RNAdraw中一个非常非常重要的特征是鼠标右键菜单打开的菜单显示对鼠标当前所指向的对象/窗口可以使用的功能列表。RNA文库（RNA
云计算大数据处理分析六大最好工具 一、概述 来自传感器、购买交易记录、网络日志等的大量数据，通常是万亿或EB的大小，如此庞大的数据，寻找一个合适处理工具非常必要，今天我们为大家分享在大数据处理分析过程中六大最好用的工具。 我们的数据来自各个方面，在面对庞大而复杂的大数据，选择一个合适的处理工具显得很有必要，工欲善其事，必须利其器，一个好的工具不仅可以使我们的工作事半功倍，也可以让我们在竞争日益激烈的云计算时代，挖掘大数据价值，及时调整战略方向。 大数据是一个含义广泛的术语，是指数据集，如此庞大而复杂的，他们需要专门设计的硬件和软件工具进行处理。该数据集通常是万亿或EB的大小。这些数据集收集自各种各样的来源：传感器、气候信息、公开的信息、如杂志、报纸、文章。大数据产生的其他例子包括购买交易记录、网络日志、病历、事监控、视频和图像档案、及大型电子商务。大数据分析是在研究大量的数据的过程中寻找模式，相关性和其他有用的信息，可以帮助企业更好地适应变化，并做出更明智的决策。 二、第一种工具：Hadoop Hadoop 是一个能够对大量数据进行分布式处理的软件框架。但是 Hadoop 是以一种可靠、高效、可伸缩的方式进行处理的。Hadoop 是可靠的，因为它假设计算元素和存储会失败，因此它维护多个工作数据副本，确保能够针对失败的节点重新分布处理。Hadoop 是高效的，因为它以并行的方式工作，通过并行处理加快处理速度。Hadoop 还是可伸缩的，能够处理 PB 级数据。此外，Hadoop 依赖于社区服务器，因此它的成本比较低，任何人都可以使用。 Hadoop是一个能够让用户轻松架构和使用的分布式计算平台。用户可以轻松地在Hadoop上开发和运行处理海量数据的应用程序。它主要有以下几个优点： ●高可靠性。Hadoop按位存储和处理数据的能力值得人们信赖。 ●高扩展性。Hadoop是在可用的计算机集簇间分配数据并完成计算任务的，这些集簇可以方便地扩 展到数以千计的节点中。 ●高效性。Hadoop能够在节点之间动态地移动数据，并保证各个节点的动态平衡，因此处理速度非 常快。 ●高容错性。Hadoop能够自动保存数据的多个副本，并且能够自动将失败的任务重新分配。 ●Hadoop带有用 Java 语言编写的框架，因此运行在 Linux 生产平台上是非常理想的。Hadoop 上的 应用程序也可以使用其他语言编写，比如 C++。 三、第二种工具：HPCC HPCC，High Performance Computing and Communications(高性能计算与通信)的缩写。1993年，由美国科学、工程、技术联邦协调理事会向国会提交了“重大挑战项目：高性能计算与通信”的报告，
一、描述统计 描述性统计是指运用制表和分类，图形以及计筠概括性数据来描述数据的集中趋势、离散趋势、偏度、峰度。 1、缺失值填充：常用方法：剔除法、均值法、最小邻居法、比率回归法、决策树法。 2、正态性检验：很多统计方法都要求数值服从或近似服从正态分布，所以之前需要进行正态性检验。常用方法：非参数检验的K-量检验、P-P图、Q-Q图、W检验、动差法。 二、假设检验 1、参数检验 参数检验是在已知总体分布的条件下（一股要求总体服从正态分布）对一些主要的参数(如均值、百分数、方差、相关系数等）进行的检验。 1）U验使用条件：当样本含量n较大时，样本值符合正态分布 2）T检验使用条件：当样本含量n较小时，样本值符合正态分布 A 单样本t检验：推断该样本来自的总体均数μ与已知的某一总体均数μ0 (常为理论值或标准值)有无差别； B 配对样本t检验：当总体均数未知时，且两个样本可以配对，同对中的两者在可能会影响处理效果的各种条件方面扱为相似； C 两独立样本t检验：无法找到在各方面极为相似的两样本作配对比较时使用。 2、非参数检验 非参数检验则不考虑总体分布是否已知，常常也不是针对总体参数，而是针对总体的某些一股性假设（如总体分布的位罝是否相同，总体分布是否正态）进行检验。 适用情况：顺序类型的数据资料，这类数据的分布形态一般是未知的。 A 虽然是连续数据，但总体分布形态未知或者非正态； B 体分布虽然正态，数据也是连续类型，但样本容量极小，如10以下； 主要方法包括：卡方检验、秩和检验、二项检验、游程检验、K-量检验等。 三、信度分析 检査测量的可信度，例如调查问卷的真实性。
大数据分析是什么？大数据分析软件有哪些？这是现在这个信息时代每一个企业管理者、经 营参与者都需要了解的。今天，小编就来针对性地总结一下，什么是大数据分析，以及2019 年主流的商业大数据分析软件。 一、大数据分析是什么 从各种各样类型的数据中，快速获得有价值信息的能力，就是大数据技术。 大数据最核心的价值就是在于对于海量数据进行存储和分析。物联网、云计算、移动互联网、车联网、手机、平板电脑、PC以及遍布地球各个角落的各种各样的传感器……我们每天能接触到数据海洋。 大数据分析的特点有以下几点：第一，数据体量巨大。从TB级别，跃升到PB级别。第二，数据类型繁多，包括网络日志、视频、图片、地理位置信息等等。第三，价值密度低。以视 频为例，连续不间断监控过程中，可能有用的数据仅仅有一两秒。第四，处理速度快。最后 这一点也是和传统的数据挖掘技术有着本质的不同。 大数据分析软件让企业能够从数据仓库获得洞察力，从而在数据驱动的业务环境中提供重要 的竞争优势。 二、 2019年大数据分析软件 1.Apache Hadoop Hadoop 是一个能够对大量数据进行分布式处理的软件框架。能够处理 PB 级数据。此外，Hadoop 依赖于社区服务器，因此它的成本比较低，任何人都可以使用。它处理速度非常快，并能够自动保存数据的多个副本。另外，带有用 Java 语言编写的框架，因此运行在 Linux 生 产平台上是非常理想的。Hadoop 上的应用程序也可以使用其他语言编写，比如 C++。 2.Storm Storm是自由的开源软件，一个分布式的、容错的实时计算系统。Storm可以非常可靠的处理 庞大的数据流，用于处理Hadoop的批量数据。 Storm很简单，支持许多种编程语言，使用 起来非常有趣。Storm由Twitter开源而来，其它知名的应用企业包括Groupon、淘宝、支付宝、阿里巴巴、乐元素、Admaster等等。应用于许多领域：实时分析、在线机器学习、不停 顿的计算、分布式RPC、 ETL等。 3.Pentaho BI
常用数据分析方法有那些 文章来源：ECP数据分析时间：2013/6/2813:35:06发布者：常用数据分析（关注：554） 标签： 本文包括： 常用数据分析方法：聚类分析、因子分析、相关分析、对应分析、回归分析、方差分析； 问卷调查常用数据分析方法：描述性统计分析、探索性因素分析、Cronbach’a 信度系数分析、结构方程模型分析(structural equations modeling)。 数据分析常用的图表方法：柏拉图(排列图)、直方图(Histogram)、散点图(scatter diagram)、鱼骨图（Ishikawa）、FMEA、点图、柱状图、雷达图、趋势图。 数据分析统计工具：SPSS、minitab、JMP。 常用数据分析方法: 1、聚类分析（Cluster Analysis） 聚类分析指将物理或抽象对象的集合分组成为由类似的对象组成的多个类的分析过程。聚类是将数据分类到不同的类或者簇这样的一个过程，所以同一个簇中的对象有很大的相似性，而不同簇间的对象有很大的相异性。聚类分析是一种探索性的分析，在分类的过程中，人们不必事先给出一个分类的标准，聚类分析能够从样本数据出发，自动进行分类。聚类分析所使用方法的不同，常常会得到不同的结论。不同研究者对于同一组数据进行聚类分析，所得到的聚类数未必一致。 2、因子分析（Factor Analysis） 因子分析是指研究从变量群中提取共性因子的统计技术。因子分析就是从大量的数据中寻找内在的联系，减少决策的困难。 因子分析的方法约有10多种，如重心法、影像分析法，最大似然解、最小平方法、阿尔发抽因法、拉奥典型抽因法等等。这些方法本质上大都属近似方法，是以相关系数矩阵为基础的，所不同的是相关系数矩阵对角线上的值，采用不同的共同性□2估值。在社会学研究中，因子分析常采用以主成分分析为基础的反覆法。 3、相关分析（Correlation Analysis） 相关分析（correlation analysis），相关分析是研究现象之间是否存在某种依存关系，并对具体有依存关系的现象探讨其相关方向以及相关程度。相关关系是一种非确定性的关系，例如，以X和Y分别记一个人的身高和体重，或分别记每公顷施肥量与每公顷小麦产量，则X与Y显然有关系，而又没有确切到可由其中的一个去精确地决定另一个的程度，这就是相关关系。 4、对应分析（Correspondence Analysis） 对应分析(Correspondence analysis)也称关联分析、R-Q型因子分析，通过分析由定性变量构成的交互汇总表来揭示变量间的联系。可以揭示同一变量的各个类别之间的差异，以及不同变量各个类别之间的对应关系。对应分析的基本思想是将一个联列表的行和列中各元素的比例结构以点的形式在较低维的空间中表示出来。
常用统计软件介绍 《概率论与数理统计》是一门实践性很强的课程。但是,目前在国内,大多侧重基本方法的介绍,而忽视了统计实验的教学。这样既不利于提高学生创新精神和实践能力,也使得这门课程的教学显得枯燥无味。为此,我们介绍一些常用的统计软件，以使学生对统计软件有初步的认识,为以后应用统计方法解决实际问题奠定初步的基础。 一、统计软件的种类 1.SAS 是目前国际上最为流行的一种大型统计分析系统，被誉为统计分析的标准软件。尽管价格不菲，SAS已被广泛应用于政府行政管理，科研，教育，生产和金融等不同领域，并且发挥着愈来愈重要的作用。目前SAS已在全球100多个国家和地区拥有29000多个客户群，直接用户超过300万人。在我国，国家信息中心，国家统计局，卫生部，中国科学院等都是SAS系统的大用户。尽管现在已经尽量“傻瓜化”，但是仍然需要一定的训练才可以使用。因此,该统计软件主要适合于统计工作者和科研工作者使用。 2.SPSS SPSS作为仅次于SAS的统计软件工具包，在社会科学领域有着广泛的应用。SPSS是世界上最早的统计分析软件，由美国斯坦福大学的三位研究生于20世纪60年代末研制。由于SPSS容易操作，输出漂亮，功能齐全，价格合理,所以很快地应用于自然科学、技术科学、社会科学的各个领域，世界上许多有影响的报刊杂志纷纷就SPSS 的自动统计绘图、数据的深入分析、使用方便、功能齐全等方面给予了高度的评价与称赞。迄今SPSS软件已有30余年的成长历史。全球
约有25万家产品用户，它们分布于通讯、医疗、银行、证券、保险、制造、商业、市场研究、科研教育等多个领域和行业，是世界上应用最广泛的专业统计软件。在国际学术界有条不成文的规定，即在国际学术交流中，凡是用SPSS软件完成的计算和统计分析，可以不必说明算法，由此可见其影响之大和信誉之高。因此,对于非统计工作者是很好的选择。 3.Excel 它严格说来并不是统计软件，但作为数据表格软件，必然有一定统计计算功能。而且凡是有Microsoft Office的计算机，基本上都装有Excel。但要注意，有时在装 Office时没有装数据分析的功能，那就必须装了才行。当然，画图功能是都具备的。对于简单分析，Excel 还算方便，但随着问题的深入，Excel就不那么“傻瓜”，需要使用函数，甚至根本没有相应的方法了。多数专门一些的统计推断问题还需要其他专门的统计软件来处理。 4.S-plus 这是统计学家喜爱的软件。不仅由于其功能齐全，而且由于其强大的编程功能，使得研究人员可以编制自己的程序来实现自己的理论和方法。它也在进行“傻瓜化”,以争取顾客。但仍然以编程方便为顾客所青睐。 5.Minitab 这个软件是很方便的功能强大而又齐全的软件，也已经“傻瓜化”，在我国用的不如SPSS与SAS那么普遍。
GPS数据处理是GPS研究的一个重要内容。目前，国际上广泛使用的GPS相对定位软件有：美国麻省理工学院（MIT）和加州大学圣地亚哥分校Scripps海洋研究所（SIO）研制的GAMIT/GLOBK，美国喷气推进实验室（JPL）研制的GIPSY/OASIS软件和瑞士BERNE大学研制的Bernese软件。选用一种好的数据处理方法和软件对GPS数据结果影响很大。在GPS静态定位领域中，几十公里以下的定位应用已经比较成熟，接收机的随机附带软件已经能够满足大多数的应用需要。但是在GPS卫星定轨以及长距离、大面积的定位应用中，如洲际板块运动监测及会战联测中，这些随机附带软件就远远不能达到要求。 Technorati 标签: GAMIT/GLOBK,GISPY/OASIS,BERNESE 近年来，GPS定位理论和软件科学的发展促进了GPS定位软件的研发，一批满足不同应用需求的GPS定位软件亦已面世。尽管不同软件在数据处理方法上各有其特点，但它们的总体结构基本上是一致的，即由数据准备、轨道计算、模型改正、数据编辑和参数估计5部分组成。 数据准备：RINREX格式的数据转换为软件特有的数据格式；剔除一些不正常的观测值（如缺伪距或某个相位数据）；根据测站的先验坐标、星历和伪距数据确定站钟偏差的先验值或站钟偏差多项式拟合系数的先验值。 轨道计算：将广播星历或精密星历改成标准轨道；如果需要改进轨道，则进行轨道积分，将卫星坐标及坐标对初始条件和其他待估参数的偏导写成列表形式。 模型改正：对观测值进行各种误差模型改正（对流层折射、潮汐、自转等）得到理论值及一阶偏导，从观测值中扣除这些理论值得到相应的验前观测残差。 数据编辑：修正相位观测值的周跳，剔除粗差。 参数估计：采用最小二乘或卡尔曼滤波估计，由编辑干净的非差观测值或双差观测值求解测站坐标、相位模糊度、（如果采用定轨或轨道松弛）卫星轨道改正值、地球自转和对流层湿分量天顶延迟等参数。 GAMIT/GLOBK GAMIT/GLOBK 软件是MIT和SIO研制的GPS综合分析软件包，可以估计卫星轨道和地面测站的三维相对位置。软件设计基于支持X－Windows的UNIX系统，现在的版本适用于Sun（OS/4，Solaris 2）、HP、IBM/RISC、DEC和基于、Intel工作站的LINUX操作系统。作为科研软件，GAMIT/GLOBK供研究和教育部门无偿使用，只需通过正式途径得到使用许可证。完全的开放性使用户可以对软件的工作原理、数据处理流程及技巧有全面的了解，这也在一定程度上促进了GAMIT/GLOBK的不断更新。 GAMIT软件处理双差观测量，采用最小二乘算法进行参数估计。采用双差观测量的优点是可以完全消除卫星钟差和接收机钟差的影响，同时也可以明显减弱诸如轨道误差、大气折射误差等系统性误差的影响。GAMIT软件主要功能和特点如下： （1）卫星轨道和地球自转参数估计；
体育舞蹈考试考生信息管理系统软件需求说明书 开发团队：智硕工作室 项目经理：武文俊 开发设计：王春磊、戴薪国 陈兆强、陈湘文 王长尧、丁廷飞
目录 1.1编写目的 (1) 1.2背景 (1) 1.3参考资料 (1) 2项目概述 (2) 2.1目标 (2) 2.2用户特点 (2) 2.3假定与约束 (2) 3 具体需求 (3) 3.1对功能的规定 (3) 3.2对性能的规定 (3) 3.2.1精度 (3) 3.2.2时间特性要求 (3) 3.2.3灵活性 (3) 4、输入输出要求 (4) 5、数据管理能力要求 (5) 6、故障处理要求 (6) 4 支持信息 (7) 4.1、软、硬件环境 (7) 4.2、接口 (7) 4.2.1、对功能的规定 (7) 4.2.2、对性能的规定 (7) 4.2.3、输入输出要求 (8) 4.3、数据管理能力要求 (10) 4.4、故障处理要求 (10)
1.1编写目的 编写“体育舞蹈考试考生信息管理系统”软件需求说明书，目的是在进行其他软件开发阶段之前完成如下的工作： ●明确用户的需求，了解用户的特点并以此设定软件开发的目标； ●明确软件的功能要求、性能要求、输入输出要求、数据管理能力要求、故障管理要求和其他专门要求。对可能涉及到的问题和用户进行充分的沟通，并在其他阶段开始之前和用户达成初步的一致，为下面将要进行的软件开发过程提供一个依据。 ●明确软件系统运行环境。 “体育舞蹈考试考生信息管理系统”软件需求说明书的预期读者是用户、开发人员和后期维护人员。 1.2背景 本项目所开发的软件系统全称为“体育舞蹈考试考生信息管理系统”。 本项目为《软件工程》课程设计大作业，同时也是为昆明学院招生就业处2014年舞蹈学专业，体育舞蹈方向招生考试而队组织开发，本项目开发主要目的为学习并熟悉软件工程项目开发流程，本项目的预期用户是昆明学院招就处工作人员。 本项目所开发游戏软件拟在Windowsxp、Windows7及以上版本操作系统下运行，拟基于C/S架构提供考生信息实时更新模式在小型局域网运行。 1.3参考资料 [1] 数据库原理与技术（SQL Server 2005）清华大学出版社 [2] Visual Basic 基础教程机械工业出版社
常用数据分析软件对比 软件优点推荐理由及学习资料 R语言 R语言与其他几种软件相比，已 经彻彻底底上升为一款相当热 门的编程软件了，当然涉及到计 算机编程可能会令不少小伙伴 们头大。这款软件强大，免费， 包罗万象，开源。是专门为统计 和数据分析开发的语言，统计前 沿的主流语言。扩展性好，丰富 的资源涵盖了多种行业数据分 析中几乎所有的方法。R与SAS 相比速度快，有大量统计分析模 块，但可扩展性稍差，昂贵。与 SPSS相比，具有复杂的用户图形 界面，简单易学，但编程十分困 难。 开源软件R是世界上最流行的数据 分析、统计计算及制图语言，几乎能 够完成任何数据处理任务，可安装并 运行于所有主流平台，为我们提供了 成千上万的专业模块和实用工具，是 从大数据中获取有用信息的绝佳工 具。本书可以说是学习R的必备教 程之一，可以让人快速进入R的世 界本书从解决实际问题入手，跳脱统 计学的理论阐述来讨论R语言及其 应用，讲解清晰透澈，极具实用性。 作者不仅高度概括了R语言的强大 功能、展示了各种实用的统计示例， 而且对于难以用传统方法分析的凌 乱、不完整和非正态的数据也给出了 完备的处理方法。这本书侧重R语 言实战，以实际项目讲解R的若干 常见应用场景。适合新手上路，回归、 方差两章展示了完整的统计分析的 过程。 《R语言实战 第二版》作者：卡巴 科弗（R obert I.Kabacoff） Eviews EViews是在Windows操作系统 中计量经济学软件里世界性领 导软件。强而有力和灵活性加上 一个便于使用者操作的界面；最 新的建模工具，快速直觉且容易 使用的软件。由于它革新的图表 使用者界面和精密的分析引擎 工具，EViews是强大，灵活性和 便于使用的功能。EViews预测 分析计量软件在科学数据分析 与评价、金融分析、经济预测、 销售预测和成本分析等领域应 用非常广泛。这也是撰写计量模 型论文最方便的软件之一。 计量经济学研究的核心是设计模型、 收集资料、估计模型、检验模型、应 用模型（结构分析、经济预测、政策 评价）。该书在数学描述方面适当淡 化，以讲清楚方法、思路为目标，不 做大量的推导和证明，重点放在如何 运用各种计量经济方法对实际的经 济问题进行分析、建模、预测、模拟 等实际操作上。该书很多内容都讲 解、总结的透彻明白，例如流量、存 量一般是否平稳等问题。 《计量经济分析方法与建模-- Eviews应用及实例(第二版)》,作者： 高铁梅 python python非常简单，非常适合人类 阅读。阅读一个良好的P ython程 序就感觉像是在读英语一样，尽 管这个英语的要求非常严格。 这本书是P andas的模块作者写的 书，被誉为P andas的最佳工具书。 P andas是python的一个数据分析 包，最初被作为金融数据分析工具而
体育舞蹈考试考生信息管理系统 软件需求说明书 开发团队：智硕工作室 项目经理：武文俊 开发设计：王春磊、戴薪国 陈兆强、陈湘文 王长尧、丁廷飞
目录 1.1编写目的 (1) 1.2背景 (1) 1.3参考资料 (1) 2项目概述 (2) 2.1目标 (2) 2.2用户特点 (2) 2.3假定与约束 (2) 3 具体需求 (3) 3.1对功能的规定 (3) 3.2对性能的规定 (3) 3.2.1精度 (3) 3.2.2时间特性要求 (3) 3.2.3灵活性 (3) 4、输入输出要求 (4) 5、数据管理能力要求 (5) 6、故障处理要求 (6) 4 支持信息 (7) 4.1、软、硬件环境 (7) 4.2、接口 (7) 4.2.1、对功能的规定 (7) 4.2.2、对性能的规定 (7) 4.2.3、输入输出要求 (8) 4.3、数据管理能力要求 (10) 4.4、故障处理要求 (10)
1.1编写目的 编写“体育舞蹈考试考生信息管理系统”软件需求说明书，目的是在进行其他软件开发阶段之前完成如下的工作： ●明确用户的需求，了解用户的特点并以此设定软件开发的目标； ●明确软件的功能要求、性能要求、输入输出要求、数据管理能力要求、故障管理要求和其他专门要求。对可能涉及到的问题和用户进行充分的沟通，并在其他阶段开始之前和用户达成初步的一致，为下面将要进行的软件开发过程提供一个依据。 ●明确软件系统运行环境。 “体育舞蹈考试考生信息管理系统”软件需求说明书的预期读者是用户、开发人员和后期维护人员。 1.2背景 本项目所开发的软件系统全称为“体育舞蹈考试考生信息管理系统”。 本项目为《软件工程》课程设计大作业，同时也是为昆明学院招生就业处2014年舞蹈学专业，体育舞蹈方向招生考试而队组织开发，本项目开发主要目的为学习并熟悉软件工程项目开发流程，本项目的预期用户是昆明学院招就处工作人员。 本项目所开发游戏软件拟在Windowsxp、Windows7及以上版本操作系统下运行，拟基于C/S架构提供考生信息实时更新模式在小型局域网运行。 1.3参考资料 [1] 数据库原理与技术（SQL Server 2005）清华大学出版社 [2] Visual Basic 基础教程机械工业出版社