Quantcast

FileNotFoundException, while file is actually available

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

FileNotFoundException, while file is actually available

Eugene Morozov-2
Hi,

I see a lot of exceptions like the following during our machine learning pipeline calculation. Spark version 2.0.2.
Sometimes it’s just few executors that fails with this message, but the job is successful.

I’d appreciate any hint you might have.
Thank you.

2017-02-05 07:56:47.022 [task-result-getter-1] WARN  o.a.spark.scheduler.TaskSetManager - Lost task 0.0 in stage 151558.0 (TID 993070, 10.61.12.43):
java.io.FileNotFoundException: File file:/path/to/file does not exist
        at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:611)
        at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:824)
        at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:601)
        at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:421)
        at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:142)
        at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:346)
        at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:769)
        at org.apache.hadoop.mapred.LineRecordReader.<init>(LineRecordReader.java:109)
        at org.apache.hadoop.mapred.TextInputFormat.getRecordReader(TextInputFormat.java:67)
        at org.apache.spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:245)
        at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:208)
        at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:101)
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319)
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:283)
        at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319)
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:283)
        at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319)
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:283)
        at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319)
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:283)
        at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:79)
        at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:47)
        at org.apache.spark.scheduler.Task.run(Task.scala:86)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)


---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: FileNotFoundException, while file is actually available

censj
If you deploy yarn model,you can used yarn logs -applicationId youApplicationId get yarn logs. You can get logs in details。
Then Looking error info,
===============================
Name: cen sujun
Mobile: 13067874572

在 2017年2月6日,05:33,Evgenii Morozov <[hidden email]> 写道:

Hi,

I see a lot of exceptions like the following during our machine learning pipeline calculation. Spark version 2.0.2.
Sometimes it’s just few executors that fails with this message, but the job is successful.

I’d appreciate any hint you might have.
Thank you.

2017-02-05 07:56:47.022 [task-result-getter-1] WARN  o.a.spark.scheduler.TaskSetManager - Lost task 0.0 in stage 151558.0 (TID 993070, 10.61.12.43):
java.io.FileNotFoundException: File file:/path/to/file does not exist
       at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:611)
       at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:824)
       at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:601)
       at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:421)
       at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:142)
       at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:346)
       at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:769)
       at org.apache.hadoop.mapred.LineRecordReader.<init>(LineRecordReader.java:109)
       at org.apache.hadoop.mapred.TextInputFormat.getRecordReader(TextInputFormat.java:67)
       at org.apache.spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:245)
       at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:208)
       at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:101)
       at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319)
       at org.apache.spark.rdd.RDD.iterator(RDD.scala:283)
       at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
       at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319)
       at org.apache.spark.rdd.RDD.iterator(RDD.scala:283)
       at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
       at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319)
       at org.apache.spark.rdd.RDD.iterator(RDD.scala:283)
       at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
       at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319)
       at org.apache.spark.rdd.RDD.iterator(RDD.scala:283)
       at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:79)
       at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:47)
       at org.apache.spark.scheduler.Task.run(Task.scala:86)
       at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
       at java.lang.Thread.run(Thread.java:745)


---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]


Loading...