Automating Software Design:Exploring and Evaluating Design
Alternatives
Abstract.Development of complex socio-technical IT systems is a very important and a relatively
new problem for Software Engineering.Traditional software development methodologies should be
revised and improved to capture properties of both human and arti?cial agents and interactions
between them.This thesis proposal aims at building a framework for the automatic selection and
evaluation of design alternatives.This is supposed to be done by(i)applying existing planning tools
to automate the generation of design alternatives,and(ii)developing methods and algorithms for
the evaluation and analysis of design alternatives based on game theory.Supporting tools will be
developed to guide their users through the software design process.
Keywords:software design,socio-technical systems,planning,game theory
1Introduction
Motivation.Development of complex socio-technical IT systems is nowadays a very important issue in Software Engineering[21].Modern IT systems involve human agents,thus,considering just system functionality is not enough–the interaction between system components and organizational/social envi-ronment should be taken into account.Because of this”human aspect”the design of socio-technical IT systems is a relatively new problem for Software Engineering.In addition,such systems are usually large-scale ones,while even for traditional large-scale systems existing software development methodologies are not very successful.Despite of the e?orts put into establishing the software development methodologies during the last decades,big projects often run out of time and budget,and are not robust and secure enough to meet continuously increasing user requirements1.To face the problem the methodologies that guide the development process since its early stages should be revised,enriched,and further developed. Moreover,the complexity of present socio-technical systems is such that to be e?ective all methodologies have to be equipped with mechanisms for automation support.
Problem.What kind of automation a designer needs?One of the proposals is to facilitate the designer’s work by automating the speci?cation re?nement process.The approach is re?ected in Model Driven Architecture(MDA)[16]which focuses on(possibly automatic)transformation from one formal system model to another.Tools supporting MDA exist and are used in the Rational Uni?ed Process for software development in UML.Yet the state-of-the-art is still not satisfactory[20].Another approach to the problem are design patterns[11]which propose to match standard documented solutions with the design problems arisen in the certain context.
Such approaches only cover part of the designer’s work,while there is another activity where the support of automation could be bene?cial as well[15]:
”Exploring alternative options is at the heart of the requirements and design processes”.
Indeed,in most current software engineering methodologies the designer has tools to report and verify the?nal choices(e.g.Goal-models in KAOS[4],UML Classes or Java code),but there is no,actually,the possibility of automatically exploring the alternatives and?nding a satisfactory one.This thesis proposal aims at exploring the problem and building the framework for the automatic selection and evaluation of design alternatives.The automated selection of alternatives at the early software development stages can 1See Standish Group reports at https://www.doczj.com/doc/5b10039486.html,/sample research/.
2Automating Software Design:Exploring and Evaluating Design Alternatives
be the most bene?cial and e?ective.The reason is that at the early stages the design space is larger,it is at these stages when most alternatives are examined(and discarded)and thus a good choice might have signi?cant economic impact.Supporting the selection of alternatives would lead to more alternatives being considered,more thorough analysis of considered alternatives and an overall more complete and trusted design.
Approach.It can be noticed that requirements–at least within the frameworks such as i*[26], Tropos[2]and the like–are conceived as networks of delegations among actors(which are organi-zational/human/software agents,positions and roles).Every delegation involves two actors,where one actor delegates to the other the delivery of a resource,the ful?llment of a goal,or the execution of a task.The delegatee can deliver/ful?ll/execute the delegated service(i.e resource,goal or task),or further delegate it,thus creating another delegation relation in the network.Intuitively,these can be seen as actions that the designer ascribes to the members of the organization and the system-to-be.Further,the task of designing such networks can be framed as a planning problem for multi-agent systems:selecting a suitable possible design corresponds to selecting a plan that satis?es the goals of the human or software agents.Many o?-the-shelves planners are available that,given the problem domain description,generate a sequence of actions(or a plan)that satisfy a set of prede?ned goals.One of the aims of this proposal is to embed existing planning tools into the framework to automate the generation of design alternatives.
Of course,the designer remains in the loop:designs generated by the planner are suggestions to be evaluated,amended and approved by the designer.The tricky point here is the solution evaluation which can be complex enough even for very experienced designers with considerable domain expertise.Indeed, a challenging characteristic of the design of socio-technical IT system is that human agents should be taken into account.They can be seen as players in a game theoretic sense as they are self-interested and rational.This means they want to minimize the load imposed personally on them,i.e.they want to reduce the number and the complexity of actions they are involved in.In a certain sense non-human agents,i.e.system components,are players as well as it is undesirable to overload them.Each player has a set of strategies he could choose from,e.g.he could decide whether to satisfy a goal himself or to pass it further to another system actor.Strategies are based on the player’s capabilities and his relations(e.g. subordination,friendship,or trust)with other human and arti?cial agents in the system.If we assume that each player ascribes a numerical weight to each possible action,then it is possible to calculate the cost of a given design alternative for the player by summing up the weights of actions in the solution he is involved in.Obviously,each rational player wants to minimize this cost,or at least he”searches for justice”,i.e.he wants the load to be more or less equally divided among all the involved actors.How to choose the”right”design alternative that will at the same time satisfy the overall system goal and be ”accepted”by all actors?A part of this research is to develop methods and algorithms for the evaluation and comparative analysis of design alternatives based on game theoretic ideas.The aim is to support the designer with the tool that will evaluate and try to optimize the solution to the design-as-planning problem.
The rest of the proposal is structured as follows:Section2overviews brie?y the areas related to the topic of the proposal;in Section3the selected approach to the problem is detailed;?nally,in Section4 already obtained results are listed and work plan for the next two years is presented.
2State-of-the-Art
Requirements Engineering and Software Design.Requirements engineering is considered to be a crucial part of software development process[23].Careful elicitation and analysis of requirements help to develop a system that meets user’s expectations,is trustful and robust.According to[23]requirements engineering involves such activities as domain analysis,elicitation,speci?cation,assessment,negotiation, documentation,and evolution.Modeling requirements to software systems and organizations in terms of goals and their interdependences has been a topic of considerable research interest during the last decades[23].A number of goal-oriented approaches for requirements representation and reasoning were
Automating Software Design:Exploring and Evaluating Design Alternatives3 introduced.For example,KAOS approach[4]supports modeling goals of di?erent types,allows to de?ne goal attributes,links between goals(e.g.to model the situation when a goal negatively or positively supports other goals),AND/OR goal re?nement links,links between goals and agents,etc.The i*model [26]o?ers primitive concepts of actors,goals and actor dependencies,which allow to model both software systems and organizations.With i*framework one can capture why the software is being developed,while the earlier approaches(e.g.Object Oriented Analysis)re?ect only what’s and how’s.
Software design is an intermediate phase between the user requirements elicitation and analysis and the system implementation.The design process results in software speci?cations which are formulated in a vocabulary understandable by programmers,while requirements are formulated in terms of objects of the real world and their interconnections,and are understandable by stakeholders.The problem of designing software that meets user requirements is addressed,for example,in[24]where a goal-oriented approach to architectural design based on the KAOS framework is proposed.The authors describe the process of deriving software speci?cations from requirements,and then building an architectural draft from functional speci?cations.The obtained architecture is then recursively re?ned to meet non-functional requirements analyzed during requirements analysis phase.
This research proposal is inspired by and was originated in Tropos project2.Tropos is an agent-oriented methodology which covers all software development phases from early and late requirements analysis through architectural and detailed design to implementation[2].It is based on i*framework with actors,goals,dependencies,plans,resources,and capabilities as basic modeling constructs.The key point in Tropos is in using the same notation through the whole software development process.During the early requirements analysis stakeholders and their intentions are identi?ed,i.e.the organizational environment of the system under development is https://www.doczj.com/doc/5b10039486.html,te requirements analysis puts the system-to-be into its operating environment and models its dependencies with the other actors of the organization. The design phase is subdivided into architectural and detailed design and is structured as follows.First, the overall architectural organization is de?ned,and the capabilities needed by the actors to ful?ll their goals and plans are identi?ed.Then,a set of agent types is de?ned together with assigning each of them one or more di?erent capabilities.Next,agents’micro level is speci?ed during the detailed design phase.Tropos framework includes not only modeling but also reasoning tools which help in requirements analysis,validation and veri?cation(e.g.goal reasoning tools,automatic veri?cation of security and trust requirements in Secure Tropos–see project homepage for the details).
Designing Social Structures.Another perspective of information systems development,inspired by the organizational theory,is re?ected in the literature.In[10]ontology for information systems is proposed which adopts i*organizational modeling framework with its actor,goal and social dependency primitives.The paper describes a number of organizational styles(e.g.joint venture,hierarchical con-tracting,etc.)and social patterns(e.g.broker,mediator,wrapper,etc.).The former describe the overall structure of the organizational context of the system or its architecture,while the latter focus on the social structures necessary to achieve one particular goal.Both organizational styles and social patterns guide the development of the organizational model for an information system.In[6]a methodology for the design of agent societies based on the type of co-ordination structure is described.Following the organizational theory,co-ordination in agent societies is divided into three types:markets,networks and hierarchies.Design steps include selecting a coordination model from ones available in the library;de-scribing interaction between the society and its environment,and behavior of the society in terms of agent roles and interaction patterns;?nally,the internal structure of agents is de?ned.
Automated Software Design.Almost?fty years ago the idea of actually deriving the code directly from the speci?cation(such as that advocated by Manna and Waldinger landmark paper)started a large program of funding for deductive program synthesis that has not gained signi?cant results in the past. The key idea of the approach is the following.A system goal together with the set of axioms are speci?ed, and then a theorem,i.e.goal of the system described in a formal speci?cation language,is proved with 2See project homepage at https://www.doczj.com/doc/5b10039486.html, for the details.
4Automating Software Design:Exploring and Evaluating Design Alternatives
the help of axioms.A program for solving the problem is extracted from the proof of the theorem.The ?eld is still fairly active and several program synthesis systems were proposed(see,e.g.[8,19]),but they are mainly domain-speci?c,require considerable expertise,and in some cases do not actually guarantee that the synthesized program will meet all requirements stated by designer[8].
Conceptually,the automatic selection of alternatives is done in deductive program synthesis:the theorem prover selects among the appropriate axioms to prove the theorem.Instead,in this proposal it is argued that the automatic selection of alternatives should and indeed can be done at earlier stages. Requirements models are by construction simpler and more abstract than software models.Therefore, techniques for automated reasoning about alternatives at the early stages of the development process may succeed where automated software synthesis has not been able to deliver.
Another approach is to facilitate the work of the designer by supporting the tedious aspects of software development by automating the speci?cation re?nement process.Such approach underlies Model Driven Architecture(MDA)[16],which focuses on(possibly automatic)transformation from one formal system model to another.MDA approach,proposed by Object Management Group,is a framework for de?ning software design methodologies.The central focus of MDA is on the model transformation,for instance from the platform-independent model of the system(PIM)to platform-speci?c models(PSMs)used for implementation purposes.Models are usually described in some formal language(e.g.UML),and the transformation is performed in accordance with the set of rules,also called mapping.Transformation could be manual,or automatic,or mixed.However,the state-of-the-art is far from being satisfactory[20].
Among the proposals on automating a software design process the one of Gamma et al.on design patterns[11]has been widely accepted.A design pattern is a solution(commonly observed from practice) to the certain problem in the certain context,so it may be thought as a problem-context-solution triple. Several design patterns can be combined to form a solution.Note that it is still the designer who makes the key decision–on what pattern to apply to the given situation.
An interesting work of Gross and Yu[14]should be mentioned here which relates the representation and analysis of non-functional requirements with software design patterns.The proposed approach or-ganizes,analyzes and re?nes non-functional requirements to provide guidance and reasoning support in applying patterns during a software system design.
AI Planning.The?eld of AI planning has been intensively developing during the last decades, and has found a number of applications(robotics,process planning,autonomous agents,etc.).Planning approach recently has proved to be applicable in the?eld of automatic Web service composition[18]. There are two basic approaches to the solution of planning problems[25].One is graph-based planning algorithms in which a compact structure,called Planning Graph,is constructed and analyzed.In the other approach the planning problem is transformed into a SAT problem and a SAT solver is used.
There exist several ways to represent the elements of a classical planning problem,i.e.the initial state of the world,the system goal,or the desired state of the world,and the possible actions system actors can perform.The widely used,and to the certain extend standard representation is PDDL(Planning Domain De?nition Language),the problem speci?cation language proposed in[13].Current PDDL version,PDDL 2.2[7]used during the last International Planning Competition3,supports many useful features,e.g. derived predicates and timed initial literals.
Design as Planning.A few works can be found which relate planning techniques with software requirements analysis and design.In[1]a program called ASAP(Automated Speci?er And Planner)is described,which automates a part of the domain-speci?c software speci?cation process.ASAP assists the designer in selecting methods for achieving user goals,discovering plans that result in undesirable outcomes,and?nding methods for preventing such outcomes.The authors describe the planner they have implemented,which combines adaptive and hierarchical planning techniques(see[1,18]for references). The problem of their approach is that the designer still performs a lot of work manually determining the 3See http://ls5-www.cs.uni-dortmund.de/～edelkamp/ipc-4/for the details.
Automating Software Design:Exploring and Evaluating Design Alternatives5 combination of goals and prohibited situations appropriate for the given application,de?ning possible start-up conditions and providing many other domain-speci?c expert knowledge.
Castillo et al.[3]present an AI planning application to assist an expert in designing control programs in the?eld of Automated Manufacturing.The system they have built integrates POCL,hierarchical and conditional planning techniques(see[3,18]for references).The authors consider standard planning approaches to be not appropriate with no ready-to-use tools for the real world,while in this research proposal the opposite point of view is advocated.Another recent application of the planning approach to requirements engineering is proposed by Gans et al.[12].Essentially,the authors map trust,con?dence and distrust described in terms of i*models[26]to delegation patterns in a work?ow model.Their approach is inspired by and implemented in ConGolog(see[18]for description and references),a logic-based planning language.However,they focus more on representing/modeling trust in social networks, than on the design automation.The authors do not go far in explaining how they exploit the planning formalism in the design process and,moreover,do not give any examples of modeling.
Game Theory.Game theory is an established discipline which deals with con?icts and cooperation among rational independent decision-makers,or players.A strategic game is de?ned by a set of players, and,for each player,a set of actions(called strategies)and a payo?function that assigns a numeric value to each of the player’s action pro?le.The theory studies various types of games:non-cooperative and cooperative games,dynamic games in which the order of players’decisions is important,games with incomplete information,etc.The key concept in classical game theory is the notion of equilibrium[17] which de?nes the strategies of each player in such a way that all players are satis?ed to a certain extent. In other words,this set of strategies is a stable state which none of the independent rational players wants to deviate from.However,this does not mean that each player maximizes his utility by choosing the equilibrium strategy;rather,we can say that by playing an equilibrium each player maximizes his utility locally,given some constraints(on the other players’actions).For example,playing the Nash equilibrium means that no player can bene?t when deviating from his equilibrium strategy given that all other players play the equilibrium.Nash equilibrium is proved to exist in mixed strategies(i.e.when a player is randomizing over several strategies),while in pure strategies it might not exist.
Game theory is applied in various areas,especially in economics(modeling markets,auctions,etc.), corporate decision making,defense strategy,telecommunications networks and many others.Among the examples are the applications of game theory to so called network games(e.g.routing,bandwidth allo-cation,etc.),see[22]for references.
Recently the idea of applying mechanism design in the area of multi-agent systems has emerged [5].Mechanism design can be viewed as a branch of game theory which intends to design systems/game environments so that certain properties are satis?ed when the equilibrium state is reached.Another name for this discipline is implementation theory[17]as it implements a particular objective despite the self-interests of individual players.However,a number of fundamental research problems should be solved[5] in order mechanism design to be actually applied to the design of complex distributed systems composed of multiple interacting agents.
3Research Contribution
3.1Problem
As it was already introduced in Section1,this thesis proposal aims at building a framework for automatic selection and evaluation of design alternatives.This is supposed to be done in two phases.The?rst is to apply existing planning tools to automate the generation of design alternatives.The second phase is to develop methods and algorithms for the alternatives evaluation and analysis based on game theoretic ideas,and,as a result,to support the designer with the tool to evaluate and optimize a solution to the design-as-planning problem.
6Automating Software Design:Exploring and Evaluating Design Alternatives
Requirements engineer/designer will be supported in the selection of the best alternative by changing the software development process as follows:
–Requirements analysis phase
?Identify system actors,goals and their properties.
?De?ne dependency relationships among actors.
–Design phase
?Automatically explore the space of design alternatives to identify delegation links and assignments of goals to actors.If no alternatives can be generated,return to the requirements analysis phase and revise the initial structure.
?With the help of supporting tools evaluate the obtained solutions.If necessary,ask for another, optimized solution.
3.2Objectives and Approach
Formalizing the design-as-planning problem.We have chosen AI planning approach to support the designer in the process of selecting the best alternative.The motivation of such choice,as it was stated in Section1,is that the problem of generating design alternatives can be naturally represented as a planning problem.The basic idea behind planning approach is to automatically determine the course of actions (i.e.a plan)needed to achieve a certain goal where an action is a transition rule from one state of the system to another[25,18].Actions are described in terms of preconditions and e?ects:if the precondition is true in the current state of the system,then the action is performed.As a consequence of an action, the system will be in a new state where the e?ect of the action is true.Thus,once we have described the initial state of the system,the goal that should be achieved(i.e.the desired?nal state of the system), and the set of possible actions that actors can perform,then the solution to the planning problem is the (not necessarily optimal)sequence of actions that allows the system to reach the desired state from the initial state.
While casting the design process as a planning problem,the following question must be addressed: which are the“actions”in software design?In Tropos approach[2]when drawing the model of a system, the designer assigns goals to actors,de?nes delegations of goals from one actor to another,and identi?es appropriate goal re?nements among the prede?ned alternative re?nements.Such actions will be used by a planner to?nd a way to ful?ll the goals of the system actors.
Planning approach requires a speci?cation language to represent the planning domain and the states of the system and its environment.Di?erent types of logic could be applied for this purpose,e.g.?rst order logic is often used to describe the planning domain with conjunctions of literals specifying the states of the system.
The language for planning domain description should provide support for specifying:
–the initial state of the system;
–the goal of the planning problem;
–the description of actions;
–the axioms of background theory.
To describe the initial state of the system,actors’and goal properties,and social relations among actors should be speci?ed.We propose to represent initial state in terms of predicates that correspond to –the possible ways of goal decomposition;
–actors’capabilities and desires to achieve a goal;
–possible delegation relations among actors.
Automating Software Design:Exploring and Evaluating Design Alternatives7 The desired state of the system(or goal of the planning problem)is described through the conjunction of predicates derived from the description of actors’desires in the initial state.Essentially,for each desired goal a predicate is added to the goal of the planning problem.
An action represents a temporal activity to accomplish an objective.The behavior of an actor can be formalized by the following actions he can perform.
Goal satisfaction.An actor can satisfy a goal only if achieving this goal is among his desires and he can actually satisfy it.The e?ect of this action is the ful?llment of the goal.
Goal delegation.An actor may have not enough capabilities to achieve his goals by himself,and so he has to delegate their satisfaction to other actors.This passage of responsibilities is performed only if the delegator wants a goal to be achieved and can depend on the delegatee to achieve it.The e?ect of this action is that the delegator does not worry any more about the satisfaction of the goal,while the delegatee takes the responsibility for the ful?llment of the goal and so it becomes his own desire to achieve it.The delegator does not care how the delegatee satis?es the goal(e.g.by his own capabilities or by further delegation),it is up to the delegatee to decide it.
AND/OR goal decomposition.As in di?erent goal-oriented modeling frameworks(e.g.as in Tropos and KAOS)two types of goal re?nement are supported:OR-decomposition,which suggests the list of alternative ways to satisfy the goal,and AND-decomposition,which re?nes the goals into subgoals which all are to be satis?ed in order to satisfy the initial goal.An actor can decompose a goal only if he wants it to be satis?ed,and only in the way which is prede?ned in the initial state of the system.The e?ect of decomposition is that the actor who re?nes the goal focuses on the ful?llment of subgoals instead of the initial goal.It is assumed that di?erent actors can decompose the same goal in di?erent ways.
In addition to actions,axioms of the planning domain can be de?ned.These are rules that hold in every state of the system and are used to complete the description of the current state.For example,to propagate goals properties along goal re?nement the following axiom is used:a goal is satis?ed if all its AND-subgoals or at least one of the OR-subgoals are satis?ed.
The proposed formalization of the design-as-planning-problem should be viewed as a starting point of our research.We foresee that the evaluation of the proposed approach on the basis of real case studies may cause the re?nement and further development of the formalization.
Applying planning.The next step,after the design problem is formalized,is to choose the”right planner”among o?-the-shelves tools available.In the last years many planners have been proposed[18]. In order to choose one of them the following requirements are considered:
–The planner should not produce redundant plans.Under non-redundant plan we mean that,by deleting an arbitrary action of the plan,the resulting plan is no more a“valid”plan(i.e.it does not allow to reach the desired state from the initial state).
–The planner should use PDDL(Planning Domain De?nition Language)since it is becoming a”stan-dard”planning language and many research groups work on its implementation.
–The language should support a number of”advanced”features(e.g.derived predicates)that are essential for implementing our planning domain,i.e.it should be at least PDDL2.2.
The?rst requirement is concerned with the optimality of the generated design decisions.We argue that it is not necessary to focus on the optimal design:human designers do not prove that their design is optimal,why should a system do it?Instead,in our framework the plan is required to be non-redundant, which guarantees at least the absence of alternative delegation paths since a plan does not contain any redundant actions.
The situation in which no solution can be found by the planner might be caused either by an error in the requirements,or by the lack of capabilities the human and software system actors were ascribed. At this point the designer needs to?nd the way to relax the initial constraints,i.e.to revise actors’desires and capabilities,and possible social dependencies.The problem with our approach is that the planner does not usually provide the point where failure occurred.Thus,our goal is to interfere into the
8Automating Software Design:Exploring and Evaluating Design Alternatives
planning process to?nd the failure cause and/or to invent heuristics to”play”with initial constraints for replanning the design till the solution is found.
Solution evaluation.Another important issue is related to the evaluation of multiple alternatives,when they are available.However,most standard planning tools do not provide all the solutions to the given problem,but they stop when the?rst plan is found.This is a natural restriction because there can be exponentially many solutions to the problem.Our approach is to treat this case by iteratively generating next alternative on the basis of the evaluation of a current one.In the following the solution evaluation issue is detailed.
Solutions to the design problem can be evaluated both from global and local perspectives,i.e.from the designer’s point of view and from the point of view of individual actors.The optimality of a solution in the global sense could be evaluated with respect to
–the length of the obtained plan;
–time required to execute the plan;
–overall plan cost(the idea of ascribing a cost to each action by each actor is described in Section1);–the degree of satisfaction of non-functional requirements in case the plan is adopted.
Regarding the?rst case,the number of actions in the obtained plan is often the criteria for the planner itself to prefer one solution to another.Thus,it can be assumed that the obtained plan is already(locally) optimal in the sense of the length minimization.
The second and third cases are closely related with the idea of plan metrics introduced in PDDL2.1 [9].Plan metrics specify the basis on which a plan is evaluated for a particular problem,and are usually numerical expressions to be minimized or maximized.Of course–and this is often the case for available planners–a planner could ignore the metrics and just evaluate a solution post hoc,which might lead to sub-optimal and possibly poor quality plans.For the minimization of time required to execute the plan the total-time variable together with the idea of durative actions can be used(the latter refers to the possibility to ascribe duration to each action of the planning domain).However,the complexity of the problem of optimizing a solution with respect to the de?ned metrics is very high and the feature is still poorly supported by the available planning tools[9].
The evaluation of design alternatives with respect to non-functional requirements satisfaction is dis-cussed in[15]:”Di?erent alternatives contribute to di?erent degrees of achievement of non-functional goals about system safety,security,performance,usability,and so forth”.The authors of this paper de?ne rules to identify application-speci?c parameters and functions to quantify impacts of di?erent explored alter-natives on goal satisfaction.
Local evaluation of the obtained plan is performed for each actor and re?ects actors’absolute(in-dividual)or relative(in comparison with other actors)assessment of the solution.For example,actor may have some upper bound on his personal load which he does not want to exceed,or he may want to minimize his individual absolute deviation from the mean utility value(where the player’s utility is de?ned as some”upper bound of outcomes”minus the personal outcome of the game,i.e.it says how much a player”saves”).
Another approach we will follow in this research to evaluate the obtained solutions is based on game theory(e.g.on the use of equilibrium concept).The substantial di?culty in applying game theoretic ideas to our problem is that all human and software agents of a socio-technical system should work as a solid mechanism satisfying the overall organizational goal.Di?erently from classical non-cooperative game theory,where all players choose their strategies independently and simultaneously before the game, in our problem actors’choices are closely interrelated.A player cannot independently change his strategy because the new action sequence will very likely be unsatisfactory,i.e.it will not be a solution anymore. Thus,to satisfy the system goals it will be necessary to impose some additional load(to compensate the one this player tries to avoid)on some other actors–and it might happen that they will not be satis?ed with their new utilities,and will try to deviate from the strategy they were imposed,and so on and so
Automating Software Design:Exploring and Evaluating Design Alternatives9 forth.We intend to build the recursive”replanning-towards-optimality”procedure and see whether it will converge to some sort of equilibrium.
A very important step of our research will be connected to the evaluation of the selected approach with the help of real-life case studies.Such case studies will serve for veri?cation of the proposed problem representation,and for testing the supporting tools based on methods and heuristics described in this proposal.
The main application of the proposed approach lies in the area of software development–mainly,it concerns the automation of passage from requirements analysis to design.Additional applications can be found in designing social structures/organizations(e.g.in business process reengineering),or in the domains where replanning and re-evaluation at runtime is needed(e.g.when queries to evaluate are assigned to the nodes of P2P database).
4Current Results and Research Plan
What is done so far:
–Initial formalization of the design-as-planning problem was done.The problem representation was translated into PDDL.
–Experiments both to evaluate the approach on simple examples and to choose the appropriate planner were conducted.
–Preliminary tool for applying planning techniques was developed.
–The?rst part of the approach(application of planning,without evaluation and optimization)was applied to Secure Tropos domain4.
–Preliminary work on iterative solution building and evaluation was done.
Research plan:
–Spring2006–Summer2006:
?Explore solution evaluation issues in the light of game theory.Assess the possibility to use mech-anism design.
?Explore other evaluation techniques,and the case of the absence of solution.
?Planning:on the basis of case studies enrich formalization,assess di?erent planners from the point of view of correctness and performance,try enhanced planning techniques(e.g.planning with duration/metrics support).
–Summer2006–Autumn2006:
?Automation framework:provide tool(s).
–Winter2007–Summer2007:
?Consider large real-life case studies to assess/verify the approach.
?Provide description of the(enriched)methodology.
–In parallel with other activities:
?Consider framework applications:P2P databases,secure systems design,BPR,etc.
–Autumn2007:
?Write the thesis.
4The resulting paper is under review for CAiSE’2006.
10Automating Software Design:Exploring and Evaluating Design Alternatives
References
1.J.S.Anderson and S.Fickas.A proposed perspective shift:viewing speci?cation design as a planning problem.
In IWSSD’89:5th Int.workshop on Software speci?cation and design,pages177–184,1989.
2.P.Bresciani,P.Giorgini,F.Giunchiglia,J.Mylopoulos,and A.Perini.TROPOS:An Agent-Oriented Software
Development Methodology.JAAMAS,8(3):203–236,2004.
3.L.Castillo,J.Fdez-Olivares,and A.Gonzlez.Integrating hierarchical and conditional planning techniques
into a software design process for automated manufacturing.In ICAPS2003,Workshop on Planning under Uncertainty and Incomplete Information,pages28–39,2003.
4. A.Dardenne,A.van Lamsweerde,and S.Fickas.Goal-directed requirements acquisition.Science of Computer
Programming,20:3–50,1993.
5.R.K.Dash,N.R.Jennings,and https://www.doczj.com/doc/5b10039486.html,putational mechanism design:A call to arms.IEEE
Intelligent Systems,18(6):40–47,Jan./Feb.2003.
6.V.Dignum and F.Dignum.Modelling agent societies:Co-ordination frameworks and institutions.In Por-
tuguese Conference on Arti?cial Intelligence,pages191–204,2001.
7.S.Edelkamp and J.Ho?mann.Pddl2.2:The language for the classical part of the4th international planning
competition.Technical Report195,University of Freiburg,2004.
8.T.Ellman.Speci?cation and synthesis of hybrid automata for physics-based animation.In ASE,pages80–93,
2003.
9.M.Fox and D.Long.Pddl2.1:An extension to pddl for expressing temporal planning domains.J.Artif.
Intell.Res.(JAIR),20:61–124,2003.
10. A.Fuxman,P.Giorgini,M.Kolp,and https://www.doczj.com/doc/5b10039486.html,rmation systems as social structures.In Proc.of
the2nd Int.Conf.on Formal Ontology in Inform.Sys.(FOIS’01),pages10–21.ACM Press,2001.
11. E.Gamma,R.Helm,R.Johnson,and J.Vlissides.Design Patterns:Elements of Reusable Object-Oriented
Software.Addison-Wesley,1995.
12.G.Gans,M.Jarke,S.Kethers,and https://www.doczj.com/doc/5b10039486.html,kemeyer.Modeling the impact of trust and distrust in agent
networks.In Proc.of AOIS’01,pages45–58,2001.
13.M.Ghallab,A.Howe,C.Knoblock,D.McDermott,A.Ram,M.Veloso,D.Weld,and D.Wilkins.PDDL–
The Planning Domain De?nition Language.In Proc.of AIPS’98,1998.
14. D.Gross and E.S.K.Yu.From non-functional requirements to design through patterns.Requirements
Engineering,6(1):18–36,2001.
15. E.Letier and A.van Lamsweerde.Reasoning about partial goal satisfaction for requirements and design
engineering.SIGSOFT Softw.Eng.Notes,29(6):53–62,2004.
16.Object Management Group.Model Driven Architecture(MDA).https://www.doczj.com/doc/5b10039486.html,/docs/ormsc/01-07-
01.pdf,July2001.
17.M.J.Osborne and A.Rubinstein.A Course in Game Theory.MIT Press,1994.
18.J.Peer.Web Service Composition as AI Planning–a Survey.Technical report,University of St.Gallen,
2005.
19.S.Roach and J.Baalen.Automated procedure construction for deductive synthesis.Automated Software
Engineering,12(4):393–414,October2005.
20.R.K.Runde and K.St?len.What is model driven architecture?Technical Report UIO-IFI-RR304,Depart-
ment of Informatics,University of Oslo,March2003.
21.I.Sommerville.Software engineering(7th ed.).Addison-Wesley,2004.
22. https://www.doczj.com/doc/5b10039486.html,work games.In Proceedings of the Annual ACM Symposium on Theory of Computing,2004.
23. A.van Lamsweerde.Requirements engineering in the year00:a research perspective.In Proc.of ICSE’00,
pages5–19.ACM,2000.
24. A.van Lamsweerde.From system goals to software architecture.In SFM,pages25–43,2003.
25. D.S.Weld.Recent Advances in AI Planning.AI Magazine,20(2):93–123,1999.
26. E.S.-K.Yu.Modelling strategic relationships for process reengineering.PhD thesis,University of Toronto,
1996.
大数据分析的六大工具介绍 2016年12月 一、概述 来自传感器、购买交易记录、网络日志等的大量数据，通常是万亿或EB的大小，如此庞大的数据，寻找一个合适处理工具非常必要，今天我们为大家分学在大数据处理分析过程中六大最好用的工具。 我们的数据来自各个方面，在面对庞大而复杂的大数据，选择一个合适的处理工具显得很有必要，工欲善其事，必须利其器，一个好的工具不仅可以使我们的工作事半功倍，也可以让我们在竞争日益激烈的云计算时代，挖掘大数据价值，及时调整战略方向。 大数据是一个含义广泛的术语，是指数据集，如此庞大而复杂的，他们需要专门设il?的硬件和软件工具进行处理。该数据集通常是万亿或EB的大小。这些数据集收集自各种各样的来源:传感器、气候信息、公开的信息、如杂志、报纸、文章。大数据产生的其他例子包括购买交易记录、网络日志、病历、事监控、视频和图像档案、及大型电子商务。大数据分析是在研究大量的数据的过程中寻找模式, 相关性和其他有用的信息，可以帮助企业更好地适应变化，并做出更明智的决策。 二.第一种工具:Hadoop Hadoop是一个能够对大量数据进行分布式处理的软件框架。但是Hadoop是 以一种可黑、高效、可伸缩的方式进行处理的。Hadoop是可靠的，因为它假设计算元素和存储会失败，因此它维护多个工作数据副本，确保能够针对失败的节点重新分布处理。Hadoop 是高效的，因为它以并行的方式工作，通过并行处理加快处理速度。Hadoop还是可伸缩的，能够处理PB级数据。此外，Hadoop依赖于社区服务器，因此它的成本比较低，任何人都可以使用。
Hadoop是一个能够让用户轻松架构和使用的分布式计算平台。用户可以轻松地 在Hadoop上开发和运行处理海量数据的应用程序。它主要有以下儿个优点: ,高可黑性。Hadoop按位存储和处理数据的能力值得人们信赖。，高扩展性。Hadoop是 在可用的计?算机集簇间分配数据并完成讣算任务 的，这些集簇可以方便地扩展到数以千计的节点中。 ，高效性。Hadoop能够在节点之间动态地移动数据，并保证各个节点的动 态平衡，因此处理速度非常快。 ，高容错性。Hadoop能够自动保存数据的多个副本，并且能够自动将失败 的任务重新分配。 ,Hadoop带有用Java语言编写的框架，因此运行在Linux生产平台上是非 常理想的。Hadoop上的应用程序也可以使用其他语言编写，比如C++。 第二种工具:HPCC HPCC, High Performance Computing and Communications（高性能计?算与通信）的缩写° 1993年，山美国科学、工程、技术联邦协调理事会向国会提交了“重大挑战项 U：高性能计算与通信”的报告，也就是被称为HPCC计划的报告，即美国总统科学战略项U ,其U的是通过加强研究与开发解决一批重要的科学与技术挑战 问题。HPCC是美国实施信息高速公路而上实施的计?划，该计划的实施将耗资百亿 美元，其主要U标要达到:开发可扩展的计算系统及相关软件，以支持太位级网络 传输性能，开发千兆比特网络技术，扩展研究和教育机构及网络连接能力。
一、基因芯片： 1、基因芯片综合分析软件。 ArrayVision 7.0 一种功能强大的商业版基因芯片分析软件，不仅可以进行图像分析，还可以进行数据处理，方便protocol的管理功能强大，商业版正式版：6900美元。 Arraypro 4.0 Media Cybernetics公司的产品，该公司的gelpro, imagepro一直以精确成为同类产品中的佼佼者，相信arraypro也不会差。 phoretix™ Array Nonlinear Dynamics公司的基因片综 合分析软件。 J-express 挪威Bergen大学编写，是一个用JAVA语言写的应用程序，界面清晰漂亮，用来分析微矩阵（microarray）实验获得的基因表达数据，需要下载安装JAVA运行环境JRE1.2后(5.1M)后，才能运行。 2、基因芯片阅读图像分析软件 ScanAlyze 2.44 ，斯坦福的基因芯片基因芯片阅读软件，进行微矩阵荧光图像分析，包括半自动定义格栅与像素点分析。输出为分隔的文本格式，可很容易地转化为任何数据库。
3、基因芯片数据分析软件 Cluster 斯坦福的对大量微矩阵数据组进行各种簇（Cluster)分析与其它各种处理的软件。 SAM Significance Analysis of Microarrays 的缩写，微矩阵显著性分析软件，EXCEL软件的插件，由Stanford大学编制。4．基因芯片聚类图形显示 TreeView 1.5 斯坦福开发的用来显示Cluster软件分析的图形化结果。现已和Cluster成为了基因芯片处理的标准软件。 FreeView 是基于JAVA语言的系统树生成软件，接收Cluster生成的数据，比Treeview增强了某些功能。 5．基因芯片引物设计 Array Designer 2.00 DNA微矩阵（microarray）软件，批量设计DNA和寡核苷酸引物工具 二、RNA二级结构。 RNA Structure 3.5 RNA Sturcture 根据最小自由能原理，将Zuker的根据RNA
以下是我在近三年做各类计量和统计分析过程中感受最深的东西，或能对大家有所帮助。当然，它不是ABC的教程，也不是细致的数据分析方法介绍，它只是“总结”和“体会”。由于我所学所做均甚杂，我也不是学统计、数学出身的，故本文没有主线，只有碎片，且文中内容仅为个人观点，许多论断没有数学证明，望统计、计量大牛轻拍。 于我个人而言，所用的数据分析软件包括EXCEL、SPSS、STATA、EVIEWS。在分析前期可以使用EXCEL进行数据清洗、数据结构调整、复杂的新变量计算（包括逻辑计算）；在后期呈现美观的图表时，它的制图制表功能更是无可取代的利器；但需要说明的是，EXCEL毕竟只是办公软件，它的作用大多局限在对数据本身进行的操作，而非复杂的统计和计量分析，而且，当样本量达到“万”以上级别时，EXCEL的运行速度有时会让人抓狂。 SPSS是擅长于处理截面数据的傻瓜统计软件。首先，它是专业的统计软件，对“万”甚至“十万”样本量级别的数据集都能应付自如；其次，它是统计软件而非专业的计量软件，因此它的强项在于数据清洗、描述统计、假设检验（T、F、卡方、方差齐性、正态性、信效度等检验）、多元统计分析（因子、聚类、判别、偏相关等）和一些常用的计量分析（初、中级计量教科书里提到的计量分析基本都能实现），对于复杂的、前沿的计量分析无能为力；第三，SPSS主要用于分析截面数据，在时序和面板数据处理方面功能了了；最后，SPSS兼容菜单化和编程化操作，是名副其实的傻瓜软件。 STATA与EVIEWS都是我偏好的计量软件。前者完全编程化操作，后者兼容菜单化和编程化操作；虽然两款软件都能做简单的描述统计，但是较之 SPSS差了许多；STATA与EVIEWS都是计量软件，高级的计量分析能够在这两个软件里得到实现；STATA的扩展性较好，我们可以上网找自己需要的命令文件（.ado文件），不断扩展其应用，但EVIEWS 就只能等着软件升级了；另外，对于时序数据的处理，EVIEWS较强。 综上，各款软件有自己的强项和弱项，用什么软件取决于数据本身的属性及分析方法。EXCEL适用于处理小样本数据，SPSS、 STATA、EVIEWS可以处理较大的样本；EXCEL、SPSS适合做数据清洗、新变量计算等分析前准备性工作，而STATA、EVIEWS在这方面较差；制图制表用EXCEL；对截面数据进行统计分析用SPSS，简单的计量分析SPSS、STATA、EVIEWS可以实现，高级的计量分析用 STATA、EVIEWS，时序分析用EVIEWS。 关于因果性 做统计或计量，我认为最难也最头疼的就是进行因果性判断。假如你有A、B两个变量的数据，你怎么知道哪个变量是因（自变量），哪个变量是果（因变量）？ 早期，人们通过观察原因和结果之间的表面联系进行因果推论，比如恒常会合、时间顺序。但是，人们渐渐认识到多次的共同出现和共同缺失可能是因果关系，也可能是由共同的原因或其他因素造成的。从归纳法的角度来说，如果在有A的情形下出现B，没有A的情形下就没有B，那么A很可能是B的原因，但也可能是其他未能预料到的因素在起作用，所以，在进行因果判断时应对大量的事例进行比较，以便提高判断的可靠性。 有两种解决因果问题的方案：统计的解决方案和科学的解决方案。统计的解决方案主要指运用统计和计量回归的方法对微观数据进行分析，比较受干预样本与未接受干预样本在效果指标（因变量）上的差异。需要强调的是，利用截面数据进行统计分析，不论是进行均值比较、频数分析，还是方差分析、相关分析，其结果只是干预与影响效果之间因果关系成立的必要条件而非充分条件。类似的，利用截面数据进行计量回归，所能得到的最多也只是变量间的数量关系；计量模型中哪个变量为因变量哪个变量为自变量，完全出于分析者根据其他考虑进行的预设，与计量分析结果没有关系。总之，回归并不意味着因果关系的成立，因果关系的判定或推断必须依据经过实践检验的相关理论。虽然利用截面数据进行因果判断显得勉强，但如果研究者掌握了时间序列数据，因果判断仍有可为，其
常用统计软件介绍
常用统计软件介绍 《概率论与数理统计》是一门实践性很强的课程。但是,目前在国内,大多侧重基本方法的介绍,而忽视了统计实验的教学。这样既不利于提高学生创新精神和实践能力,也使得这门课程的教学显得枯燥无味。为此,我们介绍一些常用的统计软件，以使学生对统计软件有初步的认识,为以后应用统计方法解决实际问题奠定初步的基础。 一、统计软件的种类 1.SAS 是目前国际上最为流行的一种大型统计分析系统，被誉为统计分析的标准软件。尽管价格不菲，SAS已被广泛应用于政府行政管理，科研，教育，生产和金融等不同领域，并且发挥着愈来愈重要的作用。目前SAS已在全球100多个国家和地区拥有29000多个客户群，直接用户超过300万人。在我国，国家信息中心，国家统计局，卫生部，中国科学院等都是SAS系统的大用户。尽管现在已经尽量“傻瓜化”，但是仍然需要一定的训练才可以使用。因此,该统计软件主要适合于统计工作者和科研工作者使用。 2.SPSS SPSS作为仅次于SAS的统计软件工具包，在社会科学领域有着广泛的应用。SPSS是世界上最早的统计分析软件，由美国斯坦福大学的三位研究生于20世纪60年代末研制。由于SPSS容易操作，输出漂亮，功能齐全，价格合理,所以很快地应用于自然科学、技术科学、社会科学的各个领域，世界上许多有影响的报刊杂志纷纷就SPSS 的自动统计绘图、数据的深入分析、使用方便、功能齐全等方面给予了高度的评价与称赞。迄今SPSS软件已有30余年的成长历史。全球
约有25万家产品用户，它们分布于通讯、医疗、银行、证券、保险、制造、商业、市场研究、科研教育等多个领域和行业，是世界上应用最广泛的专业统计软件。在国际学术界有条不成文的规定，即在国际学术交流中，凡是用SPSS软件完成的计算和统计分析，可以不必说明算法，由此可见其影响之大和信誉之高。因此,对于非统计工作者是很好的选择。 3.Excel 它严格说来并不是统计软件，但作为数据表格软件，必然有一定统计计算功能。而且凡是有Microsoft Office的计算机，基本上都装有Excel。但要注意，有时在装 Office时没有装数据分析的功能，那就必须装了才行。当然，画图功能是都具备的。对于简单分析，Excel 还算方便，但随着问题的深入，Excel就不那么“傻瓜”，需要使用函数，甚至根本没有相应的方法了。多数专门一些的统计推断问题还需要其他专门的统计软件来处理。 4.S-plus 这是统计学家喜爱的软件。不仅由于其功能齐全，而且由于其强大的编程功能，使得研究人员可以编制自己的程序来实现自己的理论和方法。它也在进行“傻瓜化”,以争取顾客。但仍然以编程方便为顾客所青睐。 5.Minitab 这个软件是很方便的功能强大而又齐全的软件，也已经“傻瓜化”，在我国用的不如SPSS与SAS那么普遍。
Chapter4 Introduction to Analysis-of-Variance Procedures Chapter T able of Contents 52Chapter4.Introduction to Analysis-of-Variance Procedures SAS OnlineDoc?:Version8 Chapter4 Introduction to Analysis-of-Variance Procedures 54Chapter4.Introduction to Analysis-of-Variance Procedures The following section presents an overview of some of the fundamental features of analysis of variance.Subsequent sections describe how this analysis is performed with procedures in SAS/STAT software.For more detail,see the chapters for the individual procedures.Additional sources are described in the“References”section on page61. De?nitions Analysis of variance(ANOV Ais a technique for analyzing experimental data in which one or more response(or dependent or simply Yvariables are measured un-der various conditions identi?ed by one or more classi?cation variables.The com-binations of levels for the classi?cation variables form the cells of the experimental design for the data.For example,an experiment may measure weight change(the dependent variablefor men and women who participated in three different weight-loss programs.The six cells of the design are formed by the six combinations of sex (men,womenand program(A,B,C.
大数据可视化分析平台 一、背景与目标 基于邳州市电子政务建设的基础支撑环境，以基础信息资源库(人口库、法人库、宏观经济、地理库)为基础，建设融合业务展示系统，提供综合信息查询展示、信息简报呈现、数据分析、数据开放等资源服务应用。实现市府领导及相关委办的融合数据资源视角，实现数据信息资源融合服务与创新服务，通过系统达到及时了解本市发展的综合情况，及时掌握发展动态,为政策拟定提供依据。 充分运用云计算、大数据等信息技术，建设融合分析平台、展示平台，整合现有数据资源,结合政务大数据的分析能力与业务编排展示能力，以人口、法人、地理，人口与地理,法人与地理，实现基础展示与分析,融合公安、交通、工业、教育、旅游等重点行业的数据综合分析，为城市管理、产业升级、民生保障提供有效支撑。 二、政务大数据平台 1、数据采集和交换需求:通过对各个委办局的指定业务数据进行汇聚,将分散的数据进行物理集中和整合管理，为实现对数据的分析提供数据支撑。将为跨机构的各类业务系统之间的业务协同，提供统一和集中的数据交互共享服务。包括数据交换、共享和EＴL等功能。 ２、海量数据存储管理需求：大数据平台从各个委办局的业务系统里抽取的数据量巨大,数据类型繁杂,数据需要持久化的存储和访问。不论是结构化数据、半结构化数据，还是非结构化数据,经过数据存储引擎进行建模后，持久化保存在存储系统上。存储系统要具备
高可靠性、快速查询能力。 3、数据计算分析需求：包括海量数据的离线计算能力、高效即席数据查询需求和低时延的实时计算能力。随着数据量的不断增加，需要数据平台具备线性扩展能力和强大的分析能力,支撑不断增长的数据量，满足未来政务各类业务工作的发展需要，确保业务系统的不间断且有效地工作。 4、数据关联集中需求：对集中存储在数据管理平台的数据，通过正确的技术手段将这些离散的数据进行数据关联,即:通过分析数据间的业务关系，建立关键数据之间的关联关系，将离散的数据串联起来形成能表达更多含义信息集合，以形成基础库、业务库、知识库等数据集。 5、应用开发需求:依靠集中数据集,快速开发创新应用，支撑实际分析业务需要。 6、大数据分析挖掘需求:通过对海量的政务业务大数据进行分析与挖掘,辅助政务决策,提供资源配置分析优化等辅助决策功能， 促进民生的发展。
漏电保护器检测记录 保护设备名称设备编号额定功率 漏电保护器型号额定电流 额定漏电 动作电流 维护电工姓名电工证号 检测日期 A相对地 B 相对电C相对地 检测 结论 检测人动作电 流 动作 时间 动作 电流 动作 时间 动作 电流 动作 时间 月日 月日 月日 月日 月日 月日 月日 月日 月日 月日 月日 注：每月检测一次
进场机械设备验收表 机械编号机械名称进场日期规格型号备注 序号项目验收内容验收结果 1 整机及附属装置 2 传动系统 3 工作部分与结构 4 操作与安全装置 5 其他 验收结论： 年月日 项目所属单位 项目经理部 主管部门
工程名称机械名称 设备型号设备编号安装日期 序号验收内容验收结果1 安装场地砼硬化，机身安装稳固，设有可靠的防护棚，有安全操作规程牌，有良好 排水措施 2 锯盘护罩、分料器、防护挡板及传动部位防护罩齐全可靠 3 设备金属外壳做保护接零并连接牢固，符合要求 4 有专用开关箱并符合要求，漏电保护器匹配合理、灵敏可靠 5 开关箱距圆盘锯距离应不大于3m 6 作业场地应配有符合防火要求消防器材 验收意见： 年月日项目负责人技术负责人安装负责人机管员安全员机械操作工
工程名称机械名称 设备型号设备编号安装日期 序号验收内容验收结果1 搅拌场地混凝土硬化，机身安装稳固，设有可靠的防护棚，有安全操作规程牌，有 良好的排水措施 2 离合器、制动器灵敏可靠，各部位润化良好，运行平稳无异常 3 传动部位防护罩、料斗保险勾齐全可靠 4 钢丝绳完好并润滑良好，端部固定符合要求 5 设备金属外壳做保护接零并连接牢固，符合要求 6 有专用开关箱并符合要求，漏电保护器匹配合理、灵敏可靠。功率大于5.5KW应采 用自动开关或降压起动装置 7 作业平台稳固，操作箱箱体完好，按钮开关灵敏可靠 8 操作人员持证上岗 验收意见： 年月日项目负责人技术负责人安装负责人机管员安全员机械操作工
常用分子生物学软件 一、基因芯片： 1、基因芯片综合分析软件。 ArrayVision 一种功能强大的商业版基因芯片分析软件，不仅可以进行图像分析，还可以进行数据处理，方便protocol的管理功能强大，商业版正式版：6900美元。 Arraypro Media Cybernetics公司的产品，该公司的gelpro, imagepro一直以精确成为同类产品中的佼佼者，相信arraypro也不会差。 phoretix? Array Nonlinear Dynamics公司的基因片综合分析软件。 J-express 挪威Bergen大学编写，是一个用JAVA语言写的应用程序，界面清晰漂亮，用来分析微矩阵（microarray）实验获得的基因表达数据，需要下载安装JAVA运行环境后后，才能运行。 2、基因芯片阅读图像分析软件 ScanAlyze ，斯坦福的基因芯片基因芯片阅读软件，进行微矩阵荧光图像分析，包括半自动定义格栅与像素点分析。输出为分隔的文本格式，可很容易地转化为任何数据库。 3、基因芯片数据分析软件 Cluster 斯坦福的对大量微矩阵数据组进行各种簇（Cluster)分析与其它各种处理的软件。 SAM Significance Analysis of Microarrays 的缩写，微矩阵显著性分析软件，EXCEL软件的插件，由Stanford大学编制。 4．基因芯片聚类图形显示 TreeView 斯坦福开发的用来显示Cluster软件分析的图形化结果。现已和Cluster成为了基因芯片处理的标准软件。 FreeView 是基于JAVA语言的系统树生成软件，接收Cluster生成的数据，比Treeview增强了某些功能。 5．基因芯片引物设计 Array Designer DNA微矩阵（microarray）软件，批量设计DNA和寡核苷酸引物工具 二、RNA二级结构。 RNA Structure RNA Sturcture 根据最小自由能原理，将Zuker的根据RNA一级序列预测RNA二级结构的算法在软件上实现。预测所用的热力学数据是最近由Turner实验室获得。提供了一些模块以扩展Zuker算法的能力，使之为一个界面友好的RNA折叠程序。允许你同时打开多个数据处理窗口。主窗口的工具条提供一些基本功能：打开文件、导入文件、关闭文件、设置程序参数、重排窗口、以及即时帮助和退出程序。RNAdraw中一个非常非常重要的特征是鼠标右键菜单打开的菜单显示对鼠标当前所指向的对象/窗口可以使用的功能列表。RNA文库（RNA
云计算大数据处理分析六大最好工具 一、概述 来自传感器、购买交易记录、网络日志等的大量数据，通常是万亿或EB的大小，如此庞大的数据，寻找一个合适处理工具非常必要，今天我们为大家分享在大数据处理分析过程中六大最好用的工具。 我们的数据来自各个方面，在面对庞大而复杂的大数据，选择一个合适的处理工具显得很有必要，工欲善其事，必须利其器，一个好的工具不仅可以使我们的工作事半功倍，也可以让我们在竞争日益激烈的云计算时代，挖掘大数据价值，及时调整战略方向。 大数据是一个含义广泛的术语，是指数据集，如此庞大而复杂的，他们需要专门设计的硬件和软件工具进行处理。该数据集通常是万亿或EB的大小。这些数据集收集自各种各样的来源：传感器、气候信息、公开的信息、如杂志、报纸、文章。大数据产生的其他例子包括购买交易记录、网络日志、病历、事监控、视频和图像档案、及大型电子商务。大数据分析是在研究大量的数据的过程中寻找模式，相关性和其他有用的信息，可以帮助企业更好地适应变化，并做出更明智的决策。 二、第一种工具：Hadoop Hadoop 是一个能够对大量数据进行分布式处理的软件框架。但是 Hadoop 是以一种可靠、高效、可伸缩的方式进行处理的。Hadoop 是可靠的，因为它假设计算元素和存储会失败，因此它维护多个工作数据副本，确保能够针对失败的节点重新分布处理。Hadoop 是高效的，因为它以并行的方式工作，通过并行处理加快处理速度。Hadoop 还是可伸缩的，能够处理 PB 级数据。此外，Hadoop 依赖于社区服务器，因此它的成本比较低，任何人都可以使用。 Hadoop是一个能够让用户轻松架构和使用的分布式计算平台。用户可以轻松地在Hadoop上开发和运行处理海量数据的应用程序。它主要有以下几个优点： ●高可靠性。Hadoop按位存储和处理数据的能力值得人们信赖。 ●高扩展性。Hadoop是在可用的计算机集簇间分配数据并完成计算任务的，这些集簇可以方便地扩 展到数以千计的节点中。 ●高效性。Hadoop能够在节点之间动态地移动数据，并保证各个节点的动态平衡，因此处理速度非 常快。 ●高容错性。Hadoop能够自动保存数据的多个副本，并且能够自动将失败的任务重新分配。 ●Hadoop带有用 Java 语言编写的框架，因此运行在 Linux 生产平台上是非常理想的。Hadoop 上的 应用程序也可以使用其他语言编写，比如 C++。 三、第二种工具：HPCC HPCC，High Performance Computing and Communications(高性能计算与通信)的缩写。1993年，由美国科学、工程、技术联邦协调理事会向国会提交了“重大挑战项目：高性能计算与通信”的报告，
一、描述统计 描述性统计是指运用制表和分类，图形以及计筠概括性数据来描述数据的集中趋势、离散趋势、偏度、峰度。 1、缺失值填充：常用方法：剔除法、均值法、最小邻居法、比率回归法、决策树法。 2、正态性检验：很多统计方法都要求数值服从或近似服从正态分布，所以之前需要进行正态性检验。常用方法：非参数检验的K-量检验、P-P图、Q-Q图、W检验、动差法。 二、假设检验 1、参数检验 参数检验是在已知总体分布的条件下（一股要求总体服从正态分布）对一些主要的参数(如均值、百分数、方差、相关系数等）进行的检验。 1）U验使用条件：当样本含量n较大时，样本值符合正态分布 2）T检验使用条件：当样本含量n较小时，样本值符合正态分布 A 单样本t检验：推断该样本来自的总体均数μ与已知的某一总体均数μ0 (常为理论值或标准值)有无差别； B 配对样本t检验：当总体均数未知时，且两个样本可以配对，同对中的两者在可能会影响处理效果的各种条件方面扱为相似； C 两独立样本t检验：无法找到在各方面极为相似的两样本作配对比较时使用。 2、非参数检验 非参数检验则不考虑总体分布是否已知，常常也不是针对总体参数，而是针对总体的某些一股性假设（如总体分布的位罝是否相同，总体分布是否正态）进行检验。 适用情况：顺序类型的数据资料，这类数据的分布形态一般是未知的。 A 虽然是连续数据，但总体分布形态未知或者非正态； B 体分布虽然正态，数据也是连续类型，但样本容量极小，如10以下； 主要方法包括：卡方检验、秩和检验、二项检验、游程检验、K-量检验等。 三、信度分析 检査测量的可信度，例如调查问卷的真实性。
大数据分析是什么？大数据分析软件有哪些？这是现在这个信息时代每一个企业管理者、经 营参与者都需要了解的。今天，小编就来针对性地总结一下，什么是大数据分析，以及2019 年主流的商业大数据分析软件。 一、大数据分析是什么 从各种各样类型的数据中，快速获得有价值信息的能力，就是大数据技术。 大数据最核心的价值就是在于对于海量数据进行存储和分析。物联网、云计算、移动互联网、车联网、手机、平板电脑、PC以及遍布地球各个角落的各种各样的传感器……我们每天能接触到数据海洋。 大数据分析的特点有以下几点：第一，数据体量巨大。从TB级别，跃升到PB级别。第二，数据类型繁多，包括网络日志、视频、图片、地理位置信息等等。第三，价值密度低。以视 频为例，连续不间断监控过程中，可能有用的数据仅仅有一两秒。第四，处理速度快。最后 这一点也是和传统的数据挖掘技术有着本质的不同。 大数据分析软件让企业能够从数据仓库获得洞察力，从而在数据驱动的业务环境中提供重要 的竞争优势。 二、 2019年大数据分析软件 1.Apache Hadoop Hadoop 是一个能够对大量数据进行分布式处理的软件框架。能够处理 PB 级数据。此外，Hadoop 依赖于社区服务器，因此它的成本比较低，任何人都可以使用。它处理速度非常快，并能够自动保存数据的多个副本。另外，带有用 Java 语言编写的框架，因此运行在 Linux 生 产平台上是非常理想的。Hadoop 上的应用程序也可以使用其他语言编写，比如 C++。 2.Storm Storm是自由的开源软件，一个分布式的、容错的实时计算系统。Storm可以非常可靠的处理 庞大的数据流，用于处理Hadoop的批量数据。 Storm很简单，支持许多种编程语言，使用 起来非常有趣。Storm由Twitter开源而来，其它知名的应用企业包括Groupon、淘宝、支付宝、阿里巴巴、乐元素、Admaster等等。应用于许多领域：实时分析、在线机器学习、不停 顿的计算、分布式RPC、 ETL等。 3.Pentaho BI
常用数据分析方法有那些 文章来源：ECP数据分析时间：2013/6/2813:35:06发布者：常用数据分析（关注：554） 标签： 本文包括： 常用数据分析方法：聚类分析、因子分析、相关分析、对应分析、回归分析、方差分析； 问卷调查常用数据分析方法：描述性统计分析、探索性因素分析、Cronbach’a 信度系数分析、结构方程模型分析(structural equations modeling)。 数据分析常用的图表方法：柏拉图(排列图)、直方图(Histogram)、散点图(scatter diagram)、鱼骨图（Ishikawa）、FMEA、点图、柱状图、雷达图、趋势图。 数据分析统计工具：SPSS、minitab、JMP。 常用数据分析方法: 1、聚类分析（Cluster Analysis） 聚类分析指将物理或抽象对象的集合分组成为由类似的对象组成的多个类的分析过程。聚类是将数据分类到不同的类或者簇这样的一个过程，所以同一个簇中的对象有很大的相似性，而不同簇间的对象有很大的相异性。聚类分析是一种探索性的分析，在分类的过程中，人们不必事先给出一个分类的标准，聚类分析能够从样本数据出发，自动进行分类。聚类分析所使用方法的不同，常常会得到不同的结论。不同研究者对于同一组数据进行聚类分析，所得到的聚类数未必一致。 2、因子分析（Factor Analysis） 因子分析是指研究从变量群中提取共性因子的统计技术。因子分析就是从大量的数据中寻找内在的联系，减少决策的困难。 因子分析的方法约有10多种，如重心法、影像分析法，最大似然解、最小平方法、阿尔发抽因法、拉奥典型抽因法等等。这些方法本质上大都属近似方法，是以相关系数矩阵为基础的，所不同的是相关系数矩阵对角线上的值，采用不同的共同性□2估值。在社会学研究中，因子分析常采用以主成分分析为基础的反覆法。 3、相关分析（Correlation Analysis） 相关分析（correlation analysis），相关分析是研究现象之间是否存在某种依存关系，并对具体有依存关系的现象探讨其相关方向以及相关程度。相关关系是一种非确定性的关系，例如，以X和Y分别记一个人的身高和体重，或分别记每公顷施肥量与每公顷小麦产量，则X与Y显然有关系，而又没有确切到可由其中的一个去精确地决定另一个的程度，这就是相关关系。 4、对应分析（Correspondence Analysis） 对应分析(Correspondence analysis)也称关联分析、R-Q型因子分析，通过分析由定性变量构成的交互汇总表来揭示变量间的联系。可以揭示同一变量的各个类别之间的差异，以及不同变量各个类别之间的对应关系。对应分析的基本思想是将一个联列表的行和列中各元素的比例结构以点的形式在较低维的空间中表示出来。
常用统计软件介绍 《概率论与数理统计》是一门实践性很强的课程。但是,目前在国内,大多侧重基本方法的介绍,而忽视了统计实验的教学。这样既不利于提高学生创新精神和实践能力,也使得这门课程的教学显得枯燥无味。为此,我们介绍一些常用的统计软件，以使学生对统计软件有初步的认识,为以后应用统计方法解决实际问题奠定初步的基础。 一、统计软件的种类 1.SAS 是目前国际上最为流行的一种大型统计分析系统，被誉为统计分析的标准软件。尽管价格不菲，SAS已被广泛应用于政府行政管理，科研，教育，生产和金融等不同领域，并且发挥着愈来愈重要的作用。目前SAS已在全球100多个国家和地区拥有29000多个客户群，直接用户超过300万人。在我国，国家信息中心，国家统计局，卫生部，中国科学院等都是SAS系统的大用户。尽管现在已经尽量“傻瓜化”，但是仍然需要一定的训练才可以使用。因此,该统计软件主要适合于统计工作者和科研工作者使用。 2.SPSS SPSS作为仅次于SAS的统计软件工具包，在社会科学领域有着广泛的应用。SPSS是世界上最早的统计分析软件，由美国斯坦福大学的三位研究生于20世纪60年代末研制。由于SPSS容易操作，输出漂亮，功能齐全，价格合理,所以很快地应用于自然科学、技术科学、社会科学的各个领域，世界上许多有影响的报刊杂志纷纷就SPSS 的自动统计绘图、数据的深入分析、使用方便、功能齐全等方面给予了高度的评价与称赞。迄今SPSS软件已有30余年的成长历史。全球
约有25万家产品用户，它们分布于通讯、医疗、银行、证券、保险、制造、商业、市场研究、科研教育等多个领域和行业，是世界上应用最广泛的专业统计软件。在国际学术界有条不成文的规定，即在国际学术交流中，凡是用SPSS软件完成的计算和统计分析，可以不必说明算法，由此可见其影响之大和信誉之高。因此,对于非统计工作者是很好的选择。 3.Excel 它严格说来并不是统计软件，但作为数据表格软件，必然有一定统计计算功能。而且凡是有Microsoft Office的计算机，基本上都装有Excel。但要注意，有时在装 Office时没有装数据分析的功能，那就必须装了才行。当然，画图功能是都具备的。对于简单分析，Excel 还算方便，但随着问题的深入，Excel就不那么“傻瓜”，需要使用函数，甚至根本没有相应的方法了。多数专门一些的统计推断问题还需要其他专门的统计软件来处理。 4.S-plus 这是统计学家喜爱的软件。不仅由于其功能齐全，而且由于其强大的编程功能，使得研究人员可以编制自己的程序来实现自己的理论和方法。它也在进行“傻瓜化”,以争取顾客。但仍然以编程方便为顾客所青睐。 5.Minitab 这个软件是很方便的功能强大而又齐全的软件，也已经“傻瓜化”，在我国用的不如SPSS与SAS那么普遍。
GPS数据处理是GPS研究的一个重要内容。目前，国际上广泛使用的GPS相对定位软件有：美国麻省理工学院（MIT）和加州大学圣地亚哥分校Scripps海洋研究所（SIO）研制的GAMIT/GLOBK，美国喷气推进实验室（JPL）研制的GIPSY/OASIS软件和瑞士BERNE大学研制的Bernese软件。选用一种好的数据处理方法和软件对GPS数据结果影响很大。在GPS静态定位领域中，几十公里以下的定位应用已经比较成熟，接收机的随机附带软件已经能够满足大多数的应用需要。但是在GPS卫星定轨以及长距离、大面积的定位应用中，如洲际板块运动监测及会战联测中，这些随机附带软件就远远不能达到要求。 Technorati 标签: GAMIT/GLOBK,GISPY/OASIS,BERNESE 近年来，GPS定位理论和软件科学的发展促进了GPS定位软件的研发，一批满足不同应用需求的GPS定位软件亦已面世。尽管不同软件在数据处理方法上各有其特点，但它们的总体结构基本上是一致的，即由数据准备、轨道计算、模型改正、数据编辑和参数估计5部分组成。 数据准备：RINREX格式的数据转换为软件特有的数据格式；剔除一些不正常的观测值（如缺伪距或某个相位数据）；根据测站的先验坐标、星历和伪距数据确定站钟偏差的先验值或站钟偏差多项式拟合系数的先验值。 轨道计算：将广播星历或精密星历改成标准轨道；如果需要改进轨道，则进行轨道积分，将卫星坐标及坐标对初始条件和其他待估参数的偏导写成列表形式。 模型改正：对观测值进行各种误差模型改正（对流层折射、潮汐、自转等）得到理论值及一阶偏导，从观测值中扣除这些理论值得到相应的验前观测残差。 数据编辑：修正相位观测值的周跳，剔除粗差。 参数估计：采用最小二乘或卡尔曼滤波估计，由编辑干净的非差观测值或双差观测值求解测站坐标、相位模糊度、（如果采用定轨或轨道松弛）卫星轨道改正值、地球自转和对流层湿分量天顶延迟等参数。 GAMIT/GLOBK GAMIT/GLOBK 软件是MIT和SIO研制的GPS综合分析软件包，可以估计卫星轨道和地面测站的三维相对位置。软件设计基于支持X－Windows的UNIX系统，现在的版本适用于Sun（OS/4，Solaris 2）、HP、IBM/RISC、DEC和基于、Intel工作站的LINUX操作系统。作为科研软件，GAMIT/GLOBK供研究和教育部门无偿使用，只需通过正式途径得到使用许可证。完全的开放性使用户可以对软件的工作原理、数据处理流程及技巧有全面的了解，这也在一定程度上促进了GAMIT/GLOBK的不断更新。 GAMIT软件处理双差观测量，采用最小二乘算法进行参数估计。采用双差观测量的优点是可以完全消除卫星钟差和接收机钟差的影响，同时也可以明显减弱诸如轨道误差、大气折射误差等系统性误差的影响。GAMIT软件主要功能和特点如下： （1）卫星轨道和地球自转参数估计；
体育舞蹈考试考生信息管理系统软件需求说明书 开发团队：智硕工作室 项目经理：武文俊 开发设计：王春磊、戴薪国 陈兆强、陈湘文 王长尧、丁廷飞
目录 1.1编写目的 (1) 1.2背景 (1) 1.3参考资料 (1) 2项目概述 (2) 2.1目标 (2) 2.2用户特点 (2) 2.3假定与约束 (2) 3 具体需求 (3) 3.1对功能的规定 (3) 3.2对性能的规定 (3) 3.2.1精度 (3) 3.2.2时间特性要求 (3) 3.2.3灵活性 (3) 4、输入输出要求 (4) 5、数据管理能力要求 (5) 6、故障处理要求 (6) 4 支持信息 (7) 4.1、软、硬件环境 (7) 4.2、接口 (7) 4.2.1、对功能的规定 (7) 4.2.2、对性能的规定 (7) 4.2.3、输入输出要求 (8) 4.3、数据管理能力要求 (10) 4.4、故障处理要求 (10)
1.1编写目的 编写“体育舞蹈考试考生信息管理系统”软件需求说明书，目的是在进行其他软件开发阶段之前完成如下的工作： ●明确用户的需求，了解用户的特点并以此设定软件开发的目标； ●明确软件的功能要求、性能要求、输入输出要求、数据管理能力要求、故障管理要求和其他专门要求。对可能涉及到的问题和用户进行充分的沟通，并在其他阶段开始之前和用户达成初步的一致，为下面将要进行的软件开发过程提供一个依据。 ●明确软件系统运行环境。 “体育舞蹈考试考生信息管理系统”软件需求说明书的预期读者是用户、开发人员和后期维护人员。 1.2背景 本项目所开发的软件系统全称为“体育舞蹈考试考生信息管理系统”。 本项目为《软件工程》课程设计大作业，同时也是为昆明学院招生就业处2014年舞蹈学专业，体育舞蹈方向招生考试而队组织开发，本项目开发主要目的为学习并熟悉软件工程项目开发流程，本项目的预期用户是昆明学院招就处工作人员。 本项目所开发游戏软件拟在Windowsxp、Windows7及以上版本操作系统下运行，拟基于C/S架构提供考生信息实时更新模式在小型局域网运行。 1.3参考资料 [1] 数据库原理与技术（SQL Server 2005）清华大学出版社 [2] Visual Basic 基础教程机械工业出版社
常用数据分析软件对比 软件优点推荐理由及学习资料 R语言 R语言与其他几种软件相比，已 经彻彻底底上升为一款相当热 门的编程软件了，当然涉及到计 算机编程可能会令不少小伙伴 们头大。这款软件强大，免费， 包罗万象，开源。是专门为统计 和数据分析开发的语言，统计前 沿的主流语言。扩展性好，丰富 的资源涵盖了多种行业数据分 析中几乎所有的方法。R与SAS 相比速度快，有大量统计分析模 块，但可扩展性稍差，昂贵。与 SPSS相比，具有复杂的用户图形 界面，简单易学，但编程十分困 难。 开源软件R是世界上最流行的数据 分析、统计计算及制图语言，几乎能 够完成任何数据处理任务，可安装并 运行于所有主流平台，为我们提供了 成千上万的专业模块和实用工具，是 从大数据中获取有用信息的绝佳工 具。本书可以说是学习R的必备教 程之一，可以让人快速进入R的世 界本书从解决实际问题入手，跳脱统 计学的理论阐述来讨论R语言及其 应用，讲解清晰透澈，极具实用性。 作者不仅高度概括了R语言的强大 功能、展示了各种实用的统计示例， 而且对于难以用传统方法分析的凌 乱、不完整和非正态的数据也给出了 完备的处理方法。这本书侧重R语 言实战，以实际项目讲解R的若干 常见应用场景。适合新手上路，回归、 方差两章展示了完整的统计分析的 过程。 《R语言实战 第二版》作者：卡巴 科弗（R obert I.Kabacoff） Eviews EViews是在Windows操作系统 中计量经济学软件里世界性领 导软件。强而有力和灵活性加上 一个便于使用者操作的界面；最 新的建模工具，快速直觉且容易 使用的软件。由于它革新的图表 使用者界面和精密的分析引擎 工具，EViews是强大，灵活性和 便于使用的功能。EViews预测 分析计量软件在科学数据分析 与评价、金融分析、经济预测、 销售预测和成本分析等领域应 用非常广泛。这也是撰写计量模 型论文最方便的软件之一。 计量经济学研究的核心是设计模型、 收集资料、估计模型、检验模型、应 用模型（结构分析、经济预测、政策 评价）。该书在数学描述方面适当淡 化，以讲清楚方法、思路为目标，不 做大量的推导和证明，重点放在如何 运用各种计量经济方法对实际的经 济问题进行分析、建模、预测、模拟 等实际操作上。该书很多内容都讲 解、总结的透彻明白，例如流量、存 量一般是否平稳等问题。 《计量经济分析方法与建模-- Eviews应用及实例(第二版)》,作者： 高铁梅 python python非常简单，非常适合人类 阅读。阅读一个良好的P ython程 序就感觉像是在读英语一样，尽 管这个英语的要求非常严格。 这本书是P andas的模块作者写的 书，被誉为P andas的最佳工具书。 P andas是python的一个数据分析 包，最初被作为金融数据分析工具而
漏电保护器检测记录 表AQ-C12-08 工程名称：河间市大川世纪城商品房15 楼编号： 保护设备 塔吊设备编号01 额定功率23KW 名称 漏电保护器 DZL15CE-100/4901 额定电流100MA 额定漏电 50MA 型号动作电流 维护电工 张梦浩电工证号冀 J012012001143 姓名 A 相对地 B 相对地C相对地 检测 检测日期检测人动作动作动作动作动作动作结论 电流时间电流时间电流时间 2013 年 10 月 39 0.07S 35 0.05S 40 0.07S 合格 10 日 11 月 10 日40 0.05S 41 0.06S 42 0.06S 合格 12 月 10 日38 0.06S 42 0.07S 38 0.07S 合格 2014 年 3 月 41 0.08S 40 0.08S 41 0.08S 合格 25 日 4 月 2 5 日40 0.07S 42 0.06S 41 0.05S 合格 5 月 25 日40 0.06S 43 0.05S 40 0.04S 合格 月日 月日 月日 月日 月日 月日 注：至少每月检测一次
表AQ-C12-08 工程名称：河间市大川世纪城商品房15#楼编号： 保护设备 总配电箱设备编号02 额定功率35KW 名称 漏电保护器 DZ10CCE-250/430 额定电流200MA 额定漏电 150MA 型号动作电流 维护电工 张梦浩电工证号冀 J012012001143 姓名 A 相对地 B 相对地C相对地 检测 检测日期检测人动作动作动作动作动作动作结论 电流时间电流时间电流时间 2013 年 10 月 80 0.07S 85 0.05S 90 0.07S 合格 10 日 11 月 10 日100 0.05S 105 0.06S 95 0.06S 合格 12 月 10 日90 0.06S 100 0.07S 90 0.07S 合格 2014 年 3 月 110 0.08S 90 0.08S 110 0.08S 合格 25 日 4 月 2 5 日120 0.07S 110 0.06S 100 0.05S 合格 5 月 25 日100 0.06S 100 0.05S 120 0.04S 合格 月日 月日 月日 月日 月日 月日 注：至少每月检测一次