使用hdfs操作文件的时候遇到以下两个异常：

1.java.io.IOException: No FileSystem for scheme: hdfs

2.org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "file"

类似No FileSystem for scheme: hdfs （file）原因是jar包或打包问题。

依赖的jar包是hadoop-commons 和 hadoop-hdfs

Different JARs (hadoop-commons for LocalFileSystem, hadoop-hdfs for DistributedFileSystem) each contain a different file called org.apache.hadoop.fs.FileSystem in their META-INFO/servicesdirectory. This file lists the canonical classnames of the filesystem implementations they want to declare (This is called a Service Provider Interface implemented via java.util.ServiceLoader, see org.apache.hadoop.FileSystem line 2622).

When we use maven-assembly-plugin, it merges all our JARs into one, and all META-INFO/services/org.apache.hadoop.fs.FileSystem overwrite each-other. Only one of these files remains (the last one that was added). In this case, the FileSystem list from hadoop-commonsoverwrites the list from hadoop-hdfs, so DistributedFileSystem was no longer declared.

解决方法如下：

1.通过配置方式：

Java代码

这个解决hdfs问题
hadoopConf.set("fs.hdfs.impl", org.apache.hadoop.hdfs.DistributedFileSystem.class.getName());

这个解决本地file问题
hadoopConf.set("fs.file.impl", org.apache.hadoop.fs.LocalFileSystem.class.getName());

2.通过打包插件方式

Xml代码

<plugin>  
    <groupId>org.apache.maven.plugins</groupId>  
    <artifactId>maven-shade-plugin</artifactId>  
    <version>2.3</version>  
    <executions>  
      <execution>  
        <phase>package</phase>  
        <goals>  
          <goal>shade</goal>  
        </goals>  
        <configuration>  
          <transformers>  
            <transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>  
          </transformers>  
        </configuration>  
      </execution>  
    </executions>  
  </plugin>

hadoop UnsupportedFileSystemException 问题

解决方法如下：