hadoop 3，2024年最新大数据开发面试题2024笔试[INFO] Apache Hadoop HDFS Nati

[INFO] Apache Hadoop HDFS Native Client ................... FAILURE [ 40.658 s] [INFO] Apache Hadoop HttpFS ............................... SKIPPED [INFO] Apache Hadoop HDFS-NFS ............................. SKIPPED [INFO] Apache Hadoop HDFS-RBF ............................. SKIPPED [INFO] Apache Hadoop HDFS Project ......................... SKIPPED [INFO] Apache Hadoop YARN ................................. SKIPPED [INFO] Apache Hadoop YARN API ............................. SKIPPED [INFO] Apache Hadoop YARN Common .......................... SKIPPED [INFO] Apache Hadoop YARN Server .......................... SKIPPED [INFO] Apache Hadoop YARN Server Common ................... SKIPPED [INFO] Apache Hadoop YARN NodeManager ..................... SKIPPED [INFO] Apache Hadoop YARN Web Proxy ....................... SKIPPED [INFO] Apache Hadoop YARN ApplicationHistoryService ....... SKIPPED [INFO] Apache Hadoop YARN Timeline Service ................ SKIPPED [INFO] Apache Hadoop YARN ResourceManager ................. SKIPPED [INFO] Apache Hadoop YARN Server Tests .................... SKIPPED [INFO] Apache Hadoop YARN Client .......................... SKIPPED [INFO] Apache Hadoop YARN SharedCacheManager .............. SKIPPED [INFO] Apache Hadoop YARN Timeline Plugin Storage ......... SKIPPED [INFO] Apache Hadoop YARN TimelineService HBase Backend ... SKIPPED [INFO] Apache Hadoop YARN TimelineService HBase Common .... SKIPPED [INFO] Apache Hadoop YARN TimelineService HBase Client .... SKIPPED [INFO] Apache Hadoop YARN TimelineService HBase Servers ... SKIPPED [INFO] Apache Hadoop YARN TimelineService HBase Server 1.2 SKIPPED [INFO] Apache Hadoop YARN TimelineService HBase tests ..... SKIPPED [INFO] Apache Hadoop YARN Router .......................... SKIPPED [INFO] Apache Hadoop YARN TimelineService DocumentStore ... SKIPPED [INFO] Apache Hadoop YARN Applications .................... SKIPPED [INFO] Apache Hadoop YARN DistributedShell ................ SKIPPED [INFO] Apache Hadoop YARN Unmanaged Am Launcher ........... SKIPPED [INFO] Apache Hadoop MapReduce Client ..................... SKIPPED [INFO] Apache Hadoop MapReduce Core ....................... SKIPPED [INFO] Apache Hadoop MapReduce Common ..................... SKIPPED [INFO] Apache Hadoop MapReduce Shuffle .................... SKIPPED [INFO] Apache Hadoop MapReduce App ........................ SKIPPED [INFO] Apache Hadoop MapReduce HistoryServer .............. SKIPPED [INFO] Apache Hadoop MapReduce JobClient .................. SKIPPED [INFO] Apache Hadoop Mini-Cluster ......................... SKIPPED [INFO] Apache Hadoop YARN Services ........................ SKIPPED [INFO] Apache Hadoop YARN Services Core ................... SKIPPED [INFO] Apache Hadoop YARN Services API .................... SKIPPED [INFO] Apache Hadoop YARN Application Catalog ............. SKIPPED [INFO] Apache Hadoop YARN Application Catalog Webapp ...... SKIPPED [INFO] Apache Hadoop YARN Application Catalog Docker Image SKIPPED [INFO] Apache Hadoop YARN Application MaWo ................ SKIPPED [INFO] Apache Hadoop YARN Application MaWo Core ........... SKIPPED [INFO] Apache Hadoop YARN Site ............................ SKIPPED [INFO] Apache Hadoop YARN Registry ........................ SKIPPED [INFO] Apache Hadoop YARN UI .............................. SKIPPED [INFO] Apache Hadoop YARN CSI ............................. SKIPPED [INFO] Apache Hadoop YARN Project ......................... SKIPPED [INFO] Apache Hadoop MapReduce HistoryServer Plugins ...... SKIPPED [INFO] Apache Hadoop MapReduce NativeTask ................. SKIPPED [INFO] Apache Hadoop MapReduce Uploader ................... SKIPPED [INFO] Apache Hadoop MapReduce Examples ................... SKIPPED [INFO] Apache Hadoop MapReduce ............................ SKIPPED [INFO] Apache Hadoop MapReduce Streaming .................. SKIPPED [INFO] Apache Hadoop Distributed Copy ..................... SKIPPED [INFO] Apache Hadoop Client Aggregator .................... SKIPPED [INFO] Apache Hadoop Dynamometer Workload Simulator ....... SKIPPED [INFO] Apache Hadoop Dynamometer Cluster Simulator ........ SKIPPED [INFO] Apache Hadoop Dynamometer Block Listing Generator .. SKIPPED [INFO] Apache Hadoop Dynamometer Dist ..................... SKIPPED [INFO] Apache Hadoop Dynamometer .......................... SKIPPED [INFO] Apache Hadoop Archives ............................. SKIPPED [INFO] Apache Hadoop Archive Logs ......................... SKIPPED [INFO] Apache Hadoop Rumen ................................ SKIPPED [INFO] Apache Hadoop Gridmix .............................. SKIPPED [INFO] Apache Hadoop Data Join ............................ SKIPPED [INFO] Apache Hadoop Extras ............................... SKIPPED [INFO] Apache Hadoop Pipes ................................ SKIPPED [INFO] Apache Hadoop OpenStack support .................... SKIPPED [INFO] Apache Hadoop Amazon Web Services support .......... SKIPPED [INFO] Apache Hadoop Kafka Library support ................ SKIPPED [INFO] Apache Hadoop Azure support ........................ SKIPPED [INFO] Apache Hadoop Aliyun OSS support ................... SKIPPED [INFO] Apache Hadoop Scheduler Load Simulator ............. SKIPPED [INFO] Apache Hadoop Resource Estimator Service ........... SKIPPED [INFO] Apache Hadoop Azure Data Lake support .............. SKIPPED [INFO] Apache Hadoop Image Generation Tool ................ SKIPPED [INFO] Apache Hadoop Tools Dist ........................... SKIPPED [INFO] Apache Hadoop Tools ................................ SKIPPED [INFO] Apache Hadoop Client API ........................... SKIPPED [INFO] Apache Hadoop Client Runtime ....................... SKIPPED [INFO] Apache Hadoop Client Packaging Invariants .......... SKIPPED [INFO] Apache Hadoop Client Test Minicluster .............. SKIPPED [INFO] Apache Hadoop Client Packaging Invariants for Test . SKIPPED [INFO] Apache Hadoop Client Packaging Integration Tests ... SKIPPED [INFO] Apache Hadoop Distribution ......................... SKIPPED [INFO] Apache Hadoop Client Modules ....................... SKIPPED [INFO] Apache Hadoop Cloud Storage ........................ SKIPPED [INFO] Apache Hadoop Tencent COS Support .................. SKIPPED [INFO] Apache Hadoop Cloud Storage Project ................ SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 03:32 min [INFO] Finished at: 2023-04-03T10:59:30Z [INFO] Final Memory: 227M/3280M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.3.4:cmake-compile (cmake-compile) on project hadoop-hdfs-native-client: make failed with error code 2 -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] cwiki.apache.org/confluence/… [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :hadoop-hdfs-native-client

RPM build errors: error: Bad exit status from /var/tmp/rpm-tmp.BnnvQI (%build) Bad exit status from /var/tmp/rpm-tmp.BnnvQI (%build)

Task :hadoop-rpm FAILED

FAILURE: Build failed with an exception.

Where: Script '/ws/packages.gradle' line: 529
What went wrong: Execution failed for task ':hadoop-rpm'.

Process 'command 'rpmbuild'' finished with non-zero exit value 1

Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
Get more help at help.gradle.org

BUILD FAILED in 3m 38s 5 actionable tasks: 5 executed


可能是protobuf安装的版本有问题，我是直接复制的别人编译hadoop3.1.x版本的需要yum的一些依赖包，但是观察hadoop3.3.4源码包中**BUILDING.txt文件内容，他需要protocol版本3.7.1 不是2.5.0，所以还是需要卸载2.5.0的安装3.7.1版本**


![](https://p3-xtjj-sign.byteimg.com/tos-cn-i-73owjymdk6/0257918533964cfaacaa2bc09cf73444~tplv-73owjymdk6-jj-mark-v1:0:0:0:0:5o6Y6YeR5oqA5pyv56S-5Yy6IEAg5py65Zmo5a2m5Lmg5LmL5b-DQUk=:q75.awebp?rk3s=f64ab15b&x-expires=1773076634&x-signature=enRP9t6Id3PJCG8C%2FXWen2byhho%3D)


 解决方法

protoc --version

libprotoc 2.5.0

which protoc

/usr/local/bin/protoc

rm /usr/local/bin/protoc

rm -rf /usr/include/google/protobuf

rm -rf /usr/local/include/google/protobuf

rm -rf /usr/lib/libproto*

rm -rf /usr/local/lib/libproto*

[root@ea94ade825a6 protobuf]# make check Making check in . make[1]: Entering directory /opt/protobuf' make check-local make[2]: Entering directory /opt/protobuf' Making lib/libgmock.a lib/libgmock_main.a in gmock /bin/sh: line 0: cd: third_party/googletest/googletest: No such file or directory make[2]: *** [check-local] Error 1 make[2]: Leaving directory /opt/protobuf' make[1]: *** [check-am] Error 2 make[1]: Leaving directory /opt/protobuf' make: *** [check-recursive] Error 1

[root@ea94ade825a6 protobuf]# ./configure 有如下提示 === configuring in third_party/googletest (/opt/protobuf/third_party/googletest) configure: WARNING: no configuration information is in third_party/googletest


缺少googletest的依赖，有梯子应该就直接从google下载了，但是这里没有梯子就手动从github下载


![](https://p3-xtjj-sign.byteimg.com/tos-cn-i-73owjymdk6/9df02fb3d9a04a948b53e4166be3839b~tplv-73owjymdk6-jj-mark-v1:0:0:0:0:5o6Y6YeR5oqA5pyv56S-5Yy6IEAg5py65Zmo5a2m5Lmg5LmL5b-DQUk=:q75.awebp?rk3s=f64ab15b&x-expires=1773076634&x-signature=cuad78ymVp6onwX7w7tkPaFBCiY%3D)


 googletest对应的版本是个未发布的中间分支![](https://p3-xtjj-sign.byteimg.com/tos-cn-i-73owjymdk6/e162f5e7a5ad43e2ab4a08cd7a8746a3~tplv-73owjymdk6-jj-mark-v1:0:0:0:0:5o6Y6YeR5oqA5pyv56S-5Yy6IEAg5py65Zmo5a2m5Lmg5LmL5b-DQUk=:q75.awebp?rk3s=f64ab15b&x-expires=1773076634&x-signature=vJzkZfCh31k9GVkY8iv7pBQlvhI%3D)

wget github.com/google/goog…


解压把内容复制到 googletest 目录


![](https://p3-xtjj-sign.byteimg.com/tos-cn-i-73owjymdk6/a57965e064464bfdb6131bd5d09f0f0b~tplv-73owjymdk6-jj-mark-v1:0:0:0:0:5o6Y6YeR5oqA5pyv56S-5Yy6IEAg5py65Zmo5a2m5Lmg5LmL5b-DQUk=:q75.awebp?rk3s=f64ab15b&x-expires=1773076634&x-signature=eO%2BJYeebKVXckdMWXML524t5bwY%3D)


 重新执行安装

./autogen.sh

#./configure

make

make check

sudo make install


 make check 通过


![](https://p3-xtjj-sign.byteimg.com/tos-cn-i-73owjymdk6/a3bf2baffe604616aa4f894ce91fb2cd~tplv-73owjymdk6-jj-mark-v1:0:0:0:0:5o6Y6YeR5oqA5pyv56S-5Yy6IEAg5py65Zmo5a2m5Lmg5LmL5b-DQUk=:q75.awebp?rk3s=f64ab15b&x-expires=1773076634&x-signature=VGClutSaXRcGO%2BEeTp3lZjEqA80%3D)



![img](https://p3-xtjj-sign.byteimg.com/tos-cn-i-73owjymdk6/6a836bd51da94e31855b55b346ca5783~tplv-73owjymdk6-jj-mark-v1:0:0:0:0:5o6Y6YeR5oqA5pyv56S-5Yy6IEAg5py65Zmo5a2m5Lmg5LmL5b-DQUk=:q75.awebp?rk3s=f64ab15b&x-expires=1773076634&x-signature=%2BR%2BhaOxUHs6vZNYPehdWghR9eMk%3D)
![img](https://p3-xtjj-sign.byteimg.com/tos-cn-i-73owjymdk6/8fd2758a78b049e09aa86c4b8b34c216~tplv-73owjymdk6-jj-mark-v1:0:0:0:0:5o6Y6YeR5oqA5pyv56S-5Yy6IEAg5py65Zmo5a2m5Lmg5LmL5b-DQUk=:q75.awebp?rk3s=f64ab15b&x-expires=1773076634&x-signature=XYp81ji3Ip4YoHGZY0REqejSS9w%3D)
![img](https://p3-xtjj-sign.byteimg.com/tos-cn-i-73owjymdk6/03b599298394488db5260486e717e0d6~tplv-73owjymdk6-jj-mark-v1:0:0:0:0:5o6Y6YeR5oqA5pyv56S-5Yy6IEAg5py65Zmo5a2m5Lmg5LmL5b-DQUk=:q75.awebp?rk3s=f64ab15b&x-expires=1773076634&x-signature=TcL0zaKqYZw1cV6F2%2BRJBVwFRPc%3D)

**既有适合小白学习的零基础资料，也有适合3年以上经验的小伙伴深入学习提升的进阶课程，涵盖了95%以上大数据知识点，真正体系化！**


**由于文件比较多，这里只是将部分目录截图出来，全套包含大厂面经、学习笔记、源码讲义、实战项目、大纲路线、讲解视频，并且后续会持续更新**

**[需要这份系统化资料的朋友，可以戳这里获取](https://gitee.com/vip204888)**