I cannot read the output of a mapreduce job.
The code:
data=to.dfs(1:10)
res = mapreduce(input = data, map = function(k, v) cbind(v, 2*v))
print(res())
[1] "/tmp/Rtmpr5Xv1g/file34916a6426bf"
And then....
from.dfs(res)
Exception in thread "main" java.io.FileNotFoundException: File does not exist: /tmp/Rtmpr5Xv1g/file34916a6426bf/_logs
...
...
Finally,
hdfs.ls("/tmp/Rtmpr5Xv1g/file34916a6426bf")
permission owner group size modtime
1 -rw------- daniel supergroup 0 2013-05-13 18:24
2 drwxrwxrwt daniel supergroup 0 2013-05-13 18:23
3 -rw------- daniel supergroup 448 2013-05-13 18:24
4 -rw------- daniel supergroup 122 2013-05-13 18:23
file
1 /tmp/Rtmpr5Xv1g/file34916a6426bf/_SUCCESS
2 /tmp/Rtmpr5Xv1g/file34916a6426bf/_logs
3 /tmp/Rtmpr5Xv1g/file34916a6426bf/part-00000
4 /tmp/Rtmpr5Xv1g/file34916a6426bf/part-00001
I note that /tmp/Rtmpr5Xv1g/file34916a6426bf/_logs is a directory
Why does the program search the file "_logs" when it is a directory??????
Thanks in advance
Alfonso
I cannot read the output of a mapreduce job.
The code:
data=to.dfs(1:10)
res = mapreduce(input = data, map = function(k, v) cbind(v, 2*v))
print(res())
[1] "/tmp/Rtmpr5Xv1g/file34916a6426bf"
And then....
from.dfs(res)
Exception in thread "main" java.io.FileNotFoundException: File does not exist: /tmp/Rtmpr5Xv1g/file34916a6426bf/_logs
...
...
Finally,
hdfs.ls("/tmp/Rtmpr5Xv1g/file34916a6426bf")
permission owner group size modtime
1 -rw------- daniel supergroup 0 2013-05-13 18:24
2 drwxrwxrwt daniel supergroup 0 2013-05-13 18:23
3 -rw------- daniel supergroup 448 2013-05-13 18:24
4 -rw------- daniel supergroup 122 2013-05-13 18:23
file
1 /tmp/Rtmpr5Xv1g/file34916a6426bf/_SUCCESS
2 /tmp/Rtmpr5Xv1g/file34916a6426bf/_logs
3 /tmp/Rtmpr5Xv1g/file34916a6426bf/part-00000
4 /tmp/Rtmpr5Xv1g/file34916a6426bf/part-00001
I note that /tmp/Rtmpr5Xv1g/file34916a6426bf/_logs is a directory
Why does the program search the file "_logs" when it is a directory??????
Thanks in advance
Alfonso