Vesper Terminology » History » Version 1
jun chen, 03/10/2025 12:46 AM
1 | 1 | jun chen | # Vesper Terminology |
---|---|---|---|
2 | |||
3 | |||
4 | ## What is a job ? |
||
5 | |||
6 | A llob is a python function but decorated by the decorator @as—llob, which has no return value, it w ill not block the caller. in this decorator as—llob, |
||
7 | we could define some dependencies, it accept a function which return a list of file names. A llob w ill be send to a idle worker after |
||
8 | its depedencies was satisified. |
||
9 | |||
10 | ## What is a worker? |
||
11 | |||
12 | a worker is a prcoess run our binary verper but works in worker—mode, in this mode, it run a loop to get llob from the scheduler and execute the llob. |
||
13 | A worker could be launche in either local machine or remote machines. |
||
14 | |||
15 | ## what is the launcher? |
||
16 | |||
17 | the launcher is a tool to launch worker in local machine or cluster, currently we have local launc her and lsf launcher, |
||
18 | the local launcher will launch worker with system call, and the lsf launcher will launch worker wi th command bsub, |
||
19 | here user need specified how to select a machine with bsub. |
||
20 | |||
21 | ## what is the map reduce? |
||
22 | |||
23 | the map reduce works in the similar way of python map reduce, but run in multi machines. there is an example: |
||
24 | ``` |
||
25 | mm = MapReduce(list(range(S)), map—func, reduce—func) |
||
26 | mm.map—reduce() |
||
27 | res = mm.get() |
||
28 | ``` |
||
29 | Notice that mm.get() will block and wait the result |