> .@(
/00DArialBlackmanhh00"DTimes New Romanhh00 DCourier Newmanhh0010DArial Blackmanhh00"@.
@n?" dd@ @@``0S 1R
22+;"v 2V8<=@ACbI5O[Q.RTVW. #+)##8e9,,HH,,HH**>>
()
()5[00AA@@Z79ʚ;ʚ;g4kdkdW0ppp@<4dddd4w0hX0>0___PPT10
pp___PPT9F(I8V0?
%U5gVerification of DataDependent Properties of MPIBased Parallel Scientific SoftwareAnastasia Mironova*hS$(41ProblemIt is hard to create correct parallel programs
Concurrency adds complexity and introduces problems such as deadlock and race conditions
Nondeterminacy makes testing even less effective
Model checking techniques have been applied to concurrent systems, however
Focus on patterns of communication and not correctness of the computation
Limited experience with MPIbased programs T1Kv1Kv ObjectiveVerification of MPIbased programs using model checking and symbolic execution
Properties considered
Freedom from Deadlock
Correctness of Computation
8e1e1)&ApproachUse a model checker to explore all possible executions
Deadlock detection
Modeling MPI functions
Abstracting away unnecessary data
Computational correctness
Extend the model checker to create symbolic representations of the executions
Compare the sequential program s symbolic representation of the result to the parallel program s representation
tKP:PPPPK:/,OutlineModel checking
Modeling the program
MPI functions
Freedom from deadlock
Correctness of Computation
Carrying out the verification
Experimental resultsB%Z@Z4Z%@4><#Typical Model Checking Architecture$$(
]YCase Studies
(pMatrix Multiplication 
written by my classmates and myself as one of the assignments for the Programming Languages Course in Computer Science (CS331) at the University of Alaska, Anchorage
GaussJordan Elimination
written at the University of Massachusetts, Amherst under the guidance of Dr. Stephen Siegel
t_]63Modeling The ProgramXKey Abstractions:
Processes and communication channels
Collective MPI functions
MessagesGY84!Processes and Channels in PROMELA""(Definition of Processes:
active proc Root{& }
active proc [numprocs  1] NonRoot{& }
active proc Coordinator{& }
Definition of Channels:
/*Data channels from root to nonroot nodes*/
chan chan_from_root[nprocs]
/*Data Channels from nonroot nodes to root */
chan chan_to_root[nprocs]
/*Collective communication channels from nodes to
Coordinator */
chan chan_c[nprocs]
]PZG1D=l4G
FD+Modeling MPI Functions: Coordinator Process,,(
:6Communication Messages Messages between the coordinator and processes participating in the computation are abstracted in the following manner:
mtype = {BCAST, REDUCE, DONE},
where
BCAST is a message sent to the Coordinator process by every process participating in the computation at the beginning of all the MPI_Bcast collective routines
REDUCE is sent to the Coordinator process by every process participating in the computation at the beginning of all the MPI_Reduce collective routines
DONE is sent to the Coordinator process by processes participating in the computation upon completion of every collective operation as expected by the Coordinator processPP }GE'Modeling to Check Freedom From Deadlock(((Further abstraction of data
mtype = {BCAST, REDUCE, DONE, DATA}
Example:
Process with rank root broadcasting data to all process participating in the computation:
Original: MPI_Bcast(void *buffer, int count, MPI_Datatype datatype, int root, MPI_Comm comm)
Simplified: MPI_Bcast(int root, DATA);
 data unit is abstract.
Accounting for synchronization issues of a blocking send operation
channel!message
if
:: 1 > empty(channel)
:: 1 > skip
fiPJPDP@P X R !D@
t
[WVerifying Freedom from Deadlock (<Applying the SPIN Model Checker
No need for special property definition
Scalability
raw
utilizing optimizations of SPIN results in extended capabilitiesTKHFModeling ComputationFMPI Communication
the same as in the deadlock model
Symbolic expressions
Representation in PROMELA:
typedef Expn{
byte length;
byte array[maxExpnSize]
}P%P$P%P9P%8>kJIValidating the Results@Key Idea:
Implement both the sequential and parallel versions of the algorithm
Apply SPIN and compare the symbolic expressions produced
Conclude that the result of the parallel computations is correct if it matches the output of the sequential code.
Example: Multiplication of matrices
Symbolic computation is performed in parallel, generating a matrix of expressions on the root process
The root process does the symbolic computations sequentially
The root process loops through the two resultant structures checking that the two are the same via a set of assertions
P" P*PPYV3More Interesting Example: GaussJordan Elimination44(zDefinition: Reduced RowEchelon form
Properties of a matrix in the reduced rowechelon form:
Each row contains only zeros until the first nonzero element, which must be a 1
As the rows are followed from top to bottom, the first nonzero number occurs further to the right than in the previous row
The entries above and below the first 1 in each row must be all 0
Example:pc"
"
> DB2Matrix Multiplication vs. GaussJordan Elimination33( Why is it harder to verify the correctness of the GaussJordan Elimination?
Need to introduce branching when conditions are functions of data
The property is harder to express since there is no closed formula of the answer, again, consequence of data dependencies
lNBzMBz_Z<Example of Symbolic GaussJordan Elimination on a 2X2 matrix==(
KHDealing with Data Dependencies(
\XExperimental EvaluationProblems
Unreasonable execution time (exponential);
Insufficient memory ( large cases never run to completion);
Distinguishing feature:
The common problem of state explosion is overshadowed by issues associated with the size of the State Vector;
Experimental Results
Multiplication of Matrices
Matrix dimensions: nxn, where n = 2, 3, & , 7
Number of processes: numprocs = 1, & , 10
GaussJordan Elimination
Sequential
Matrix sizes: mxn, where m, n = 2, 3, 4
Parallel
Matrix sizes: mxn, where m, n = 2, 3, 4
PhPPPoPPPYPPP)P
P+P hn
P@@130ConclusionsDeadlock detection
demonstrated applicability of abstractions
scaled somewhat reasonably ability to do nontrivial sizes of matrices
Computational correctness
sequential model capable of handling nontrivial sizes of matrices
using SPIN to create symbolic expressions and comparing these expressions for the parallel and sequential versions of the algorithm
Although more work needs to be done, this project served as a proof of concept.
Conference paper in progress.PuPPPrPuP 0` 33` Sf3f` 33g` f` www3PP` ZXdbmo` \ғ3y`Ӣ` 3f3ff` 3f3FKf` hk]wwwfܹ` ff>>\`Y{ff` R>& {p_/̴>?" dd@,?" dd@ " @ ` n?" dd@ @@``PR @ ` `p>>
(
6 " `}
T Click to edit Master title style!
!$
0 " `
RClick to edit Master text styles
Second level
Third level
Fourth level
Fifth level!
S
0L "^ `
>*
0ȓ "^
@*
0x "^ `
@*H
0h ? 3380___PPT10.apƬ Default Design
70L0 (
r
S'g
H
0h ? 33___PPT10i.zu@+D=' =
@B +
70L00(
x
c$ `}
%
x
c$@ `
H
0h ? 33___PPT10i.=+D=' =
@B +
70L0 <(
~
s*XN% `}
%
~
s*0O% `%
H
0h ? 33___PPT10i.u >e+D=' =
@B +
70L0 0(
x
c$R% `}
%
x
c$S% `%
H
0h ? 33___PPT10i.`Y+D=' =
@B +
70L00(
x
c$0d% `}
%
x
c$d% `%
H
0h ? 33___PPT10i. =+D=' =
@B +
70L0
(
x
c$j% `}
%
<n%'
CProgram
(C, MPI)
<r%&
JProgram Model
(PROMELA)
<v% 4
GModel Checker
(SPIN)
<\n%/
:Property RB
s*DW
0}%"
=abstraction 2RB
s*D RB
@
s*D& RB
@
s*D=]
IRB
s*D]
0%l ,
Cproperty verified 2
0,%I@&
^,counter example
(execution trace)
produced 2
0% C0,$
0
Builds a graph representing all possible states of the system
State explosion problem: number of states is exponential in the number of concurrent processes
Hence necessary to create an abstracted version of the program to eliminate unnecessary details wrt the property being proved&@!
0@%
,$
0
D Usually represented in temporal logic or sometimes as an automaton
,DDH
0h ? 330TIMING11.526.9T___PPT104.A+6$D' =
@B DS' =
@BA?%,( <+O%,(
<+DA' =%(D' =%(D' =A@BBBB0B%(D' =1:Bvisible*o3>+B#style.visibility<* %(DA' =%(D' =%(D' =A@BBBB0B%(D' =1:Bvisible*o3>+B#style.visibility<* %(+p+0+ 0 ++0+ 0 +
70
1(
r
S% `}
%
r
S% `V%
0@%
sTesting was performed on both of these implementations and based on the results the code was assumed to be correct.t tH
0h ? 33___PPT10i.{{+D=' =
@B +
70L00(
x
c$% `}
%
x
c$% `%
H
0h ? 33___PPT10i.c)+D=' =
@B +
70L0{s(
x
c$d% `}
%
<<% `%
0D%jFQ
;Coordinator 2
0$sP7
4Root 2
0s`
:
NonRoot[0] 2
0ps`
\NonRoot[nprocs 1] 2
0s~qe
2& 2RB
s*D3fz@RB
s*D3fz~RB
s*D3fz)qRB
s*D3fzg#RB
s*D@IPRB
s*D~RB
s*D`IqRB
s*DfmJ}RB
s*DfRB
@
s*DfIH
0h ? 33___PPT10i.+D=' =
@B +
70L0,@h(
@x
@ c$"s `}
s
@
0T)slq
; MPI_Bcast
2
@
0xs_'GF
6Proc 0 2
@
00s
'G
6Proc n 2
@
0l4s
;Coordinator 2
@
08sB'G)
6Proc 1 2
@
0;s 'G
8 & 2RB
@
s*D3f
@
0?s
5BCAST 2RB
@
s*D3fRB
@
s*D3f b
@
0tDs*J
8 & 2
@
0LHs'Gj
6Proc 0 2
@
0Ls''G
6Proc n 2
@
0Os
@v
;Coordinator 2
@
0hSsf
'GM
6Proc 1 2
@
0VsD'G+
8 & 2RB
@
s*D3f
,
@
0Zs)
4DONE 2RB
@
s*D3f,
RB
@
s*D3f,
@
0_sWw
8 & 2
@
0a4 2
dL
0dg
Fa2( 2
eL
0py6
t
3A = 2
fL
06
z
3B = 2
gL
0䆑w
>a3 2
hL
0{6
fw
>a5 2
iL
0{xw_
>a9 2
jL
0{xfw_
>a8 2
kL
0،{:
w!
>a6 2
lL
00{x_
?a10 2
mL
0{:
!
>a7 2
nL
0`s?}
?b11 2
oL
0<%O
?b12 2
pL
0%`
?b13 2
qL
0+:
P
!
?b15 2
rL
06
@~
?b14 2
sL
0:
`
!
?b16 2
tL
0(xP
_
?b18 2
uL
0p^x`
_
?b19 2
vL
0ߒs@~Z
?b17 2LB
wL
c$DLB
xL
c$DfLB
yL
c$DmLB
zL
c$DLB
{L
c$DiLB
L
c$Dp
}L
0et
<Enumeration:
~L
0U
x>Note: 0 and 1 are reserved for representing integer constants.? 2;LB
L
c$DpLB
L
c$DnH
L0h ? 33___PPT10i.u0+D=' =
@B +
70L0
p
X(
Xx
X c$L{ `}
{
X S{ `N{
"p`Pp
X
<a`
:
sequential
X
<{`0
8parallel
X
<{5
F
7compareRB
X
s*Dz 5
RB
X@
s*D #5
X
0{% @
9results 2H
X0h ? 33___PPT10i.0Uu+D=' =
@B +N
70L0MEP78(
x
c$PA `}
A
c$A `A
"p`PpLB
c$DwLB
c$Dw LB
c$DLB
c$DM LB
c$DwwLB
c$DwM w
0A
=0 2
0{X
=0 2
0ԁ*h
>2 2
0A
=0 2
0#AMG4
=2 2
0(AG
=2 2
0y
=7 2
0/A!
>12 2
0x3AX
=4 2
07AMX4
=4 2
0;A*
?10 2
0@AHV/
>5 2
0EA
=6 2
0@IAH/
=6 2
0MA
>12 2
0QAH/
>5 2
0DVAM
>28 2
0ZAHO /
>1 2LB
c$Dw
LB
c$DwLB
c$D
LB
c$DLB
!
c$Dw
wLB
"
c$Dww
#
0`ACT
=1 2
$
0eAS
=2 2
%
0iAd
> 0 2
&
0lnA
=3 2
'
0rAMB4
=0 2
(
0wAB
=0 2
)
0p{AJ
=0 2
*
0A[
> 7 2
+
0AS
=0 2
,
0LAMS4
=0 2

0Ad
> 1 2
.
0AH/
=0 2
/
0PA
=0 2
0
0AH/
=0 2
1
0AJ
=0 2
2
0,AHJ/
=1 2
3
0AH
=1 2
4
0̪AHJ/
=2 2RB
5
s*D
}
6
0AZ
]
3
&
JGaussJordan Elimination 2
7
0As&m
AOriginal matrix 2
8
0PA
<
JReduced RowEchelon form 2H
0h ? 33___PPT10i._T͓+D=' =
@B +
70L08<(
8~
8 s*A `}
A
~
8 s*\A `A
H
80h ? 33___PPT10i.x{14+D=' =
@B +1
70L000ZZw0(
~
s*A `}
A
LB
c$D]`LB
c$D]/
/
`LB
c$D`,`LB
c$D` /
`LB
c$D],]LB
c$D] 0
]
0pAY
Ga11( 2
0Ai
?a12 2
08AX
?a21 2
0xAi
?a22 2
0A/@
LCase 1: ( 2
0A.C
Ua11 = 02 2
0AC
Ua21 = 02 2
0AC
Ua12 = 02 2
0pAKC2
Ua22 = 02 2LB
c$DLB
c$DLB
c$D*LB
c$DwLB
c$D*LB
c$Dx
0xW
10 2
0xg
=0 2
0 xV
=0 2
04x"g
=0 2
0x~
LCase 2: ( 2
0xx
Ua11 = 02 2
0xx
Ua21 = 02 2
0"x xf
Ua12 = 02 2
0'x1
x
Va22 != 02 2LB
!
c$Dp..sLB
"
c$Dp1
1
sLB
#
c$Ds.sLB
$
c$Ds 1
sLB
%
c$Dp.pLB
&
c$Dp 2
p
'
0/x!
10 2
(
03x
=0 2
)
07x
=0 2
*
0H
0x
G
Va22 free2 2LB
?
c$D9
<LB
@
c$D9
0
0
<LB
A
c$D<<LB
B
c$D< 0
<LB
C
c$D9
9
LB
D
c$D9
1
9
E
0x
{
11 2
F
04x
{
=0 2
G
0$x
=0 2
H
0x
=1 2
I
0Xx
m
LCase 5: ( 2
J
0Px
Va11 != 02 2LB
K
c$Df
iLB
L
c$Df
iLB
M
c$DiWiLB
N
c$DiiLB
O
c$Df
Wf
LB
P
c$Df
f
Q
0Шx
11 2
R
0x
=0 2
S
0px
=0 2
T
0xG
Va11 != 02 2
U
0 /* Assume (a11 == 0) */
/* Add expression (a11 == 0) to the path conditions table */
&
:: 1 > /* Assume (a11 != 0) */
/* Add expression (a11 != 0) to the path conditions table */
&
fi0 2f 20
" 6
T
0yK m@
k7 Explore Both Branches
Keep a table of path conditions8 28^B
T@
6DԔ@
T
HGCY,CY,D
a11 ? 0L 2 HH
T
HKVCY,CY,2&
a11 != 0L 2 HH
T
H숇CY,CY,2D&
a11 = 0L 2 HH ^B
T
6DԔ@)H
T0h ? 33___PPT10i.+D=' =
@B +}
70
$(
r
Sy `}
y
r
SlyM `y
H
0h ? 33___PPT10i.uЋ$+D=' =
@B +
70L00(
x
c$̆ `}
x
c$ȏ `i
H
0h ? 33___PPT10i. pC+D=' =
@B +r(JN)Q/RT3 @$KL6l8\o:>VD0F0xjJ ǻYP6@
!!i_H&a1Oh+'0Php
Slide 1Anastasia MironovaAnastasia Mironova424Microsoft Office PowerPoint@@u@К\G(g %y$xx'@Arial. %2
Verification of Data."System@Arial. 2
^E.@Arial. 2
a
Dependent .@Arial. !2
(Properties of MPIa.@Arial. 2
(TE.@Arial. 2
(WBased Parallel .@Arial. $2
1Scientific Softwarea.@Arial. "2
E5Anastasia Mironova.՜.+,0
JOnscreen Show'nArialTimes New RomanCourier NewArial BlackDefault DesignhVerification of DataDependent Properties of MPIBased Parallel Scientific Software Anastasia MironovaProblem
Objective ApproachOutline$Typical Model Checking Architecture
Case StudiesModeling The Program"Processes and Channels in PROMELA,Modeling MPI Functions: Coordinator ProcessCommunication Messages(Modeling to Check Freedom From Deadlock Verifying Freedom from DeadlockModeling ComputationValidating the Results4More Interesting Example: GaussJordan Elimination3Matrix Multiplication vs. GaussJordan Elimination=Example of Symbolic GaussJordan Elimination on a 2X2 matrixDealing with Data DependenciesExperimental EvaluationConclusionsFonts UsedDesign Template
Slide Titles*_'0Anastasia MironovaAnastasia Mironova
!"#$%&'()*+,./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{}~Root EntrydO)Current UserSummaryInformation(PowerPoint Document('DocumentSummaryInformation8