PowerSI_MPI_Distributed_Simulation_Tutorial

PowerSI MPI Distributed Simulation Tutorial Product Version 16.6January 2014Document Updated on: April 2, 20142014 Cadence Design Systems, Inc. All rights reserved.Cadence Design Systems, Inc. (Cadence), 2655 Seely Ave., San Jose, CA 95134, USA.Open SystemC, Open SystemC Initiative, OSCI, SystemC, and SystemC Initiative are trademarks or registered trademarks of Open SystemC Initiative, Inc. in the United States and other countries and are used with permission.Trademarks: Trademarks and service marks of Cadence Design Systems, Inc. contained in this document are attributed to Cadence with the appropriate symbol. For queries regarding Cadence’s trademarks, contact the corporate legal department at the address shown above or call 800.862.4522.All other trademarks are the property of their respective holders.Restricted Permission: This publication is protected by copyright law and international treaties and contains trade secrets and proprietary information owned by Cadence. Unauthorized reproduction or distribution of this publication, or any portion of it, may result in civil and criminal penalties. Except as specified in this permission statement, this publication may not be copied, reproduced, modified, published, uploaded, posted, transmitted, or distributed in any way, without prior written permission from Cadence. Unless otherwise agreed to by Cadence in writing, this statement grants Cadence customers permission to print one (1) hard copy of this publication subject to the following conditions:1. The publication may be used only in accordance with a written agreement betweenCadence and its customer.2. The publication may not be modified in any way.3. Any authorized copy of the publication or portion thereof must include all originalcopyright, trademark, and other proprietary notices and this permission statement.4. The information contained in this document cannot be used in the development of likeproducts or software, whether for internal or external use, and shall not be used for thebenefit of any other party, whether or not for consideration.Disclaimer: Information in this publication is subject to change without notice and does not represent a commitment on the part of Cadence. Except as may be explicitly set forth in such agreement, Cadence does not make, and expressly disclaims, any representations or warranties as to the completeness, accuracy or usefulness of the information contained in this document. Cadence does not warrant that use of such information will not infringe any third party rights, nor does Cadence assume any liability for damages or costs of any kind that may result from use of such information.Restricted Rights: Use, duplication, or disclosure by the Government is subject to restrictionsas set forth in FAR52.227-14 and DFAR252.227-7013 et seq. or its successor.Table of Contents1Introduction (4)1.1MPI Advantages (4)1.2Environment Configuration (4)1.3Equations Applied in Distributed Computing (4)1.3.1Performance Parameters (4)1.3.2Performance Parameter for Resource Scheduling Based on CPU/Core (5)2Simple Operations (6)2.1Installing MPI Run-Time Library (6)2.1.1For Windows (6)2.1.2For Linux (6)2.2Supported Mode (6)3Running MPI Based Distributed PowerSI (7)3.1Interactive Mode (Windows Only) (7)3.2Batch Mode (8)3.2.1On Windows Platform (8)3.2.2On Linux Platform (9)3.3Using Job Scheduler Management Software (10)3.3.1Linux Version (10)3.3.2Submitting Job Using Platform LSF (10)1 IntroductionThis Tutorial introduces MPI(Message Passing Interface) based distributed simulation with PowerSI.1.1 MPI AdvantagesWith MPI based distributed simulation, user can get more correct results with much less time. T5 and T6 are applied to show performance result.•T5 – time used for solving package and circuits •T6 – total time used for this project NOTE!Pre-processing phase takes place on all computers. This time cannot be savedby distributed model, which means it is not included in T5. 1.2 Environment ConfigurationTo perform MPI based distributed simulation, the following requirements are necessary:• Slave side (remote machine)–Intel MPI run-time environment(Windows & Linux) –In interactive mode, SI Remote Service should be started •Master side (local machine) – Intel MPI run-time environment(Windows & Linux) NOTE! PowerSI should be installed in the same folder on all machines. 1.3Equations Applied in Distributed Computing 1.3.1 Performance ParametersThe following two equations are applied in distributed computing:multi PCstime with Execution PC time with Executionon Ratio Accelerati 1= i PCs with mult ution time sed x Exec Computer u PC time with ExecutionEfficiency Parallel 1 =1.3.2 Performance Parameter for Resource Scheduling Based onCPU/CoreResource Scheduled by Core means each PowerSI will only start one S600 for simulation at a time. Usually this option is used for running simulation on cluster/server farm. It cannot be used in interactive mode.esmulti Cor time with Execution Core time with Executionon Ratio Accelerati 1= ores th multi C on time wi x Executi cores used of Number Core time with ExecutionEfficiency Parallel 1 =2 Simple Operations2.1 Installing MPI Run-Time LibraryTo perform MPI simulation, MPI runtime library need to be installed first. MPI runtime library(4.1.0.028 and later version) can be downloaded from Intel website (/en-us/intel-mpi-library).2.1.1 For WindowsFollow the steps below to install MPI run-time library on Windows platform:1.Copy w_mpi-rt_p_4.1.0.028.exe to a temp folder.2.Double-click w_mpi-rt_p_4.1.0.028.exe.3.Follow the installation wizard to install the package.4.Repeat steps 1 to 3 on all the computers.NOTE! MPI run-time library path will be added to system path environment variable automatically and MPI daemon service will be started.2.1.2 For LinuxFollow the steps below to install MPI run-time library on Linux platform:1.Login as root.2.Copy l_mpi-rt_p_4.1.0.030.tgz to a temp folder.e the following command to unzip the package:tar –zxf l_mpi-rt_p_4.1.0.030.tgz4.Change current folder to l_mpi-rt_p_4.1.0.030.e the following command to install the package:./install.sh6.Follow the installation wizard to install the package.7.(Optional) Add MPI bin folder to PATH environment.NOTE! On Linux, make sure the installation folder can be accessed by all computers.2.2 Supported ModeSupported simulation mode: Extraction Mode and Spatial Mode.Supported frequency sweep mode: AFS(Extraction Mode only) and Linaer/Log mode.3 Running MPI Based Distributed PowerSI3.1 Interactive Mode (Windows Only)In interactive mode, SI Service Manager is still needed to get the server list. For more informationabout how to start SI Remote Service on remote machine, please refer to PowerSI_UG.To perform MPI based distributed PowerSI,1.Open .spd file.2.SelectSetup > Server SetupThe Server Setup For PowerSI window opens.3.Check the Use MPI option.4.Input the domain, user name and password in corresponding fields.This information is required. The fields cannot be left blank.5.Click OK after all settings.6.Click the Start Simulation button.New PowerSI instances will start in batch mode on local and remote machines. Do not clickRun again.NOTE! •In interactive mode, current PowerSI instance will not participate in simulation. It is used to start batch mode MPI based PowerSI only •New instances of PowerSI will start in batch mode on both local machine and remote machines•The simulation in process cannot be stopped by clicking Stop button •Since PowerSI runs in batch mode, all the results will be output to harddisk only after simulation is completed. After that user should load resultto output window manually. No result can be checked during simulation•The password saved in pwd.txt must be in plain text format, which isrequired by MPI. The files containing user name and password will bedeleted automatically when current PowerSI exits normally•Resource schedule is based on current user setting in registry instead of oncore•Interactive mode is not recommended for running MPI based PowerSI 3.2 Batch Mode3.2.1 On Windows PlatformUsing MPI to perform distributed simulation in batch mode doesn’t need SI Service Manager torun on remote server. It is required that the system provides a "single sign-on" where onepassword for a user is shared among all the computers.1.Prepare machine fileUse a file to list the names of machines to launch on. For mpiexec option, it is–machinefile filename“machinefile” lists all the machines involved in simulation. The format of machine file shouldbe: <host name>:<number of processes>Example:The following machine files in Format 1 and Format 2 are the same:Format 1host1host1host2Format 2host1:2host2NOTE! Local machine must be selected in machine list and in the first row of the file. In the example above, if user launches five PowerSIs, the first two are on host1, the third one on host2, and the 4th and 5th on host1.2.(Optional) Prepare user name and password fileMPI reads the account and password from the specified file: the account is on the first line and the password on the second. This is for option “-pwdfile filename”.3.Run MPI based Distributed PowerSImpiexec -machinefile filename -delegate/-pwdfile filename [-wdir workdirectory] –n x powersi -b [-bycore] example.spdNOTE! •The options within [ ] are optional.•Use either “-delegate” or “-pwdfile filename” to set the type of authorization. “-delegate” is preferred.•-delegateo Enable the domain-based authorization with the delegation ability.•-pwdfile filenameo Enable the password-based authorization.o Read the account and password from the file specified: the account is on the firstline and the password on the second.•-bycoreo If selected, the resource is scheduled by core.o If not selected, the resource is scheduled by default setting in registry.3.2.2 On Linux PlatformUsing MPI to perform distributed simulation in batch mode doesn’t need SI Service Manager torun on remote server. It is required that the system provides a "single sign-on" where onepassword for a user is shared among all the computers.1.Prepare machine fileUse a file to list the names of machines to launch on. For mpirun option, it is-machinefile filename“machinefile” lists all the machines involved in simulation. The format of machine file shouldbe: <host name>:<number of processes>Example:The following machine files in Format 1 and Format 2 are the same:Format 1host1host1host2Format 2host1:2host2NOTE! Local machine must be selected in machine list and in the first row of the file. In the example above, if user starts five PowerSIs, the first two PowerSIs start on host1, the third PowerSI starts on host2, the 4th and 5th PowerSIs start on host1.2.Run MPI based Distributed PowerSI/path/mpirun -machinefile file [-bootstrap rsh] [-wdir workdirectory] -n x powersi -b -NoUI [-bycore] example.spd• Option -bootstrap rsh is used when rsh is used for remote connection• Option -NoUI must be added for Linux platform• -bycore means the resource is scheduled by core-If this option is not selected, the resource is scheduled by default setting in registry3.3 Using Job Scheduler Management Software3.3.1 Linux VersionThere are lots of Job Scheduler applications for Windows and Linux. Among them, Platform LSF,PBS Professional, Sun Grid Engine (SGE)/Oracle Grid Engine (OGE), and TORQUE ResourceManager are commonly used.Different applications work in different ways for job management. A script file based on the JobScheduler software should be generated before using. In the script file, user can specify thenumber of PowerSIs to start, resource requirement and max running time. Read the referencemanual of the software you use.Not all the computers are the same in the clusters. The computers may have different OS anddifferent hardware configuration. PowerSI only supports Windows, SUSE Linux/Redhat Linux.You should specify resource requirement in the script file, including: OS name, OS version,memory requirement, CPU speed, CPU type (Intel or AMD), queue name, and so on.Resource requirement affects the time waiting in queue. Estimate the time before submitting a job.If the job runs longer than the specified time, it will be killed. Run time is the actual time that thejob runs once on a CPU, and the pending time is not counted.NOTE! Most clusters assign resource by core instead of by computer. So the new option, -bycore, must be added to PowerSI. Also –b and –NoUI must be added.3.3.2 Submitting Job Using Platform LSF1.Prepare script file. The following example shows how to create a script file.#!/bin/bash#BSUB -n 8#BSUB -R "OSNAME==Linux && ( SPEED>=2500 ) && ( OSREL==EE60 ||OSREL==EE58 || OSREL==EE63 ) && SFIARCH==OPT64 && mem>=32000"#BSUB -q lnx64#BSUB -W 1:40cd my_working_directory/path/mpirun powersi -b -NoUI -bycore example.spd2.Submit job using bsub command.bsub < scriptfilee bjobs to check job status.bjobse bkill command to kill submitted job.bkill jobid。