MATLAB PARALLEL COMPUTING TOOLBOX - S Uživatelská příručka Strana 201
- Strana / 656
- Tabulka s obsahem
- ŘEŠENÍ PROBLÉMŮ
- KNIHY
Hodnocené. / 5. Na základě hodnocení zákazníků
Run mapreduce on a Hadoop Cluster
6-61
Run mapreduce on a Hadoop Cluster
In this section...
“Cluster Preparation” on page 6-61
“Output Format and Order” on page 6-61
“Calculate Mean Delay” on page 6-61
Cluster Preparation
Before you can run mapreduce on a Hadoop
®
cluster, make sure that the cluster and
client machine are properly configured. Consult your system administrator, or see
“Configure a Hadoop Cluster”.
Output Format and Order
When running mapreduce on a Hadoop cluster with binary output (the default), the
resulting KeyValueDatastore points to Hadoop Sequence files, instead of binary MAT-
files as generated by mapreduce in other environments. For more information, see the
'OutputType' argument description on the mapreduce reference page.
When running mapreduce on a Hadoop cluster, the order of the key-value pairs in the
output is different compared to running mapreduce in other environments. If your
application depends on the arrangement of data in the output, you must sort the data
according to your own requirements.
Calculate Mean Delay
This example shows how modify the MATLAB example for calculating mean airline
delays to run on a Hadoop cluster.
First, you must set environment variables and cluster properties as appropriate for your
specific Hadoop configuration. See your system administrator for the values for these and
other properties necessary for submitting jobs to your cluster.
setenv('HADOOP_HOME','/share/hadoop/a2.2.0');
cluster = parallel.cluster.Hadoop;
cluster.HadoopProperties('mapred.job.tracker') = 'hadoophost1:50031';
cluster.HadoopProperties('fs.default.name') = 'hdfs://hadoophost2:8020';
- Parallel Computing Toolbox™ 1
- User's Guide 1
- How to Contact MathWorks 2
- Revision History 3
- Contents 5
- Program Independent Jobs 11
- Program Communicating Jobs 12
- GPU Computing 12
- Objects — Alphabetical List 14
- Functions — Alphabetical List 14
- Glossary 14
- Getting Started 15
- 1 Getting Started 16
- Partition Large Data Sets 19
- Run a Batch Job 22
- Run a Batch Parallel Loop 23
- Parallel for-Loops (parfor) 29
- Introduction to parfor 30
- Create a parfor-Loop 32
- Create a parfor-Loop 33
- 2 Parallel for-Loops (parfor) 34
- parfor Limitations 39
- Using Objects in parfor-Loops 41
- Handle Classes 41
- Nested Functions 43
- Nested Loops 43
- Nested spmd Statements 45
- Break and Return Statements 45
- P-Code Scripts 45
- Unambiguous Variable Names 47
- Transparency 47
- Loop Variable 53
- Sliced Variables 55
- Broadcast Variables 59
- Reduction Variables 60
- Reduction Variables 65
- Temporary Variables 67
- Temporary Variables 69
- Improving parfor Performance 70
- Improving parfor Performance 71
- Parallel Pools 72
- Parallel Pools 73
- Start a Parallel Pool 73
- When to Use spmd 80
- Define an spmd Statement 81
- Display Output 83
- Introduction to Composites 84
- Distribute Arrays 88
- Create Codistributed Arrays 89
- P>> help magic 97
- P>> x = pi 97
- P>> x = labindex 98
- P>> all = numlabs 98
- P>> getLocalPart(I) 102
- P>> pmode exit 103
- Parallel Command Window 104
- Click tiling icon 105
- Select layout 105
- Select vertical 106
- Drag to adjust 106
- Click tabbed output 107
- Select only two tabs 107
- Multiple labs 108
- Using Graphics in pmode 112
- Connectivity Testing 113
- Hostname Resolution 113
- Socket Connections 113
- Introduction 116
- Nondistributed Arrays 116
- Codistributed Arrays 118
- A = zeros(80, 1000); 119
- D = codistributed(A) 119
- Local Arrays 124
- Restoring the Full Array 127
- 2-Dimensional Distribution 130
- Parallelizing a for-Loop 134
- plot(1:numDataSets, res); 135
- Programming Overview 141
- 6 Programming Overview 142
- Toolbox and Server Components 143
- Life Cycle of a Job 147
- Parallel Preferences 152
- Parallel Preferences 153
- Clusters and Cluster Profiles 154
- Job Monitor 169
- Job Monitor 171
- Programming Tips 172
- Programming Tips 173
- Using the pause Function 175
- Interrupting a Job 175
- Speeding Up a Job 175
- Control Random Number Streams 177
- Client and GPU 179
- Worker CPU and Worker GPU 181
- Profiling Parallel Code 182
- Viewing Parallel Profile Data 183
- Profiling Parallel Code 185
- Benchmarking Performance 191
- Troubleshooting and Debugging 192
- Caused by: 195
- Start Parallel Pool 197
- Compare Parallel mapreduce 197
- See Also 199
- Cluster Preparation 201
- Output Format and Order 201
- Calculate Mean Delay 201
- Related Examples 203
- More About 203
- 7 Program Independent Jobs 212
- MJS Cluster 217
- Properties 217
- Share Code with the Workers 224
- Share Code with the Workers 227
- Overview 229
- Client node 231
- MATLAB client 231
- Scheduler 231
- MATLAB Worker Decode Function 235
- Program Communicating Jobs 249
- 8 Program Communicating Jobs 250
- Code in the Client 251
- 9 GPU Computing 260
- Establish Arrays on a GPU 261
- Establish Arrays on a GPU 265
- 100 100 265
- Supported MATLAB Code 273
- Run CUDA or PTX Code on GPU 278
- Create a CUDAKernel Object 279
- Run CUDA or PTX Code on GPU 281
- Compile a GPU MEX-File 291
- Comparison to a CUDA Kernel 291
- Constructor 302
- Description 302
- codistributed 303
- Properties 304
- Composite 306
- CUDAKernel 307
- distributed 311
- GPUDevice 314
- Constructors 319
- Container Hierarchy 319
- parallel.Cluster 321
- HPC Server 322
- PBS Pro and TORQUE 322
- MJS Jobs 332
- CJS Jobs 332
- RemoteClusterAccess 341
- Examples 344
- Input Arguments 346
- arrayfun 349
- Arguments 352
- Create gpuArray False Matrix 430
- Create gpuArray Inf Matrix 484
- Output Arguments 513
- mpiprofile 519
- C Syntax 524
- Create gpuArray NaN Matrix 549
- Create gpuArray Rand Matrix 605
- Create gpuArray True Matrix 641
- Glossary 653
- Glossary-5 655
Související produkty a manuály pro Software MATLAB PARALLEL COMPUTING TOOLBOX - S
Software MATLAB EMBEDDED IDE LINK 4 - FOR USE WITH TEXAS INSTRUMENTS CODE COMPOSER STUDIO Uživatelská příručka
(112 stránky)
(112 stránky)
Software MATLAB POLYSPACE 7 Instalační příručka
(618 stránky)
(618 stránky)
Software MATLAB EMBEDDED IDE LINK 4 - FOR USE WITH TEXAS INSTRUMENTS CODE COMPOSER STUDIO Specifikace
(126 stránky)
(126 stránky)
Software MATLAB BUILDER EX 1 Uživatelská příručka
(164 stránky)
(164 stránky)
© 2020, manymanuals.cz. Všechna práva vyhrazena. | 0.024 s |
Manymanuals.com
Manymanuals.de
Manymanuals.fr
Manymanuals.it
Manymanuals.pl
Manymanuals.cz
Manymanuals.es
Manymanuals-pt.com
Komentáře k této Příručce