💖💖作者：IT跃迁谷毕设展 💙💙个人简介：曾长期从事计算机专业培训教学，本人也热爱上课教学，语言擅长Java、微信小程序、Python、Golang、安卓Android等，开发项目包括大数据、深度学习、网站、小程序、安卓、算法。平常会做一些项目定制化开发、代码讲解、答辩教学、文档编写、也懂一些降重方面的技巧。平常喜欢分享一些自己开发中遇到的问题的解决办法，也喜欢交流技术，大家有技术代码这一块的问题可以问我！ 💛💛想说的话：感谢大家的关注与支持！ 💜💜
💕💕文末获取源码
基于Hadoop+Spark的帕金森病数据可视化分析系统-功能介绍
基于Hadoop+Spark的帕金森病数据可视化分析系统是一个专门针对医疗大数据领域的综合性分析平台，采用当前最主流的大数据技术架构进行设计与实现。系统以帕金森病语音数据为核心研究对象，通过Hadoop分布式存储框架和Spark大数据计算引擎，对包含22个语音特征指标的帕金森病数据集进行分析。技术架构上，系统后端采用Python+Django框架或Java+Spring Boot框架构建RESTful API接口，前端使用Vue+ElementUI+Echarts技术栈打造数据可视化界面，数据存储基于MySQL数据库，并通过HDFS实现大数据的分布式存储管理。系统核心功能涵盖四大分析维度：①数据集整体健康度与患者群体画像分析，通过对患者与健康人群的样本均衡性分析和关键指标描述性统计，建立全面的数据认知基础；②帕金森病核心语音特征差异性对比分析，深入探索音高特征、声音频率微扰、振幅微扰以及嗓音质量等关键指标在患者与健康人群间的显著差异；③特征关联性挖掘与关键指标识别，通过相关性分析和基于算法的特征重要性排序，科学识别对帕金森病诊断最具价值的语音特征组合；④非线性动力学特征深度探索，利用RPDE、DFA、D2、PPE等前沿指标从系统混沌和信号复杂度角度提供诊断依据。
基于Hadoop+Spark的帕金森病数据可视化分析系统-选题背景意义
帕金森病作为全球第二大神经退行性疾病，其发病态势日趋严峻。中国流行病学调查数据显示，2021年帕金森病患者人数已超过500万，中国有着数量最大的PD患者群体，发病率也高于全球平均。世界卫生组织预测，到2040年，包括帕金森病和阿尔茨海默病在内的神经退行性疾病将成为全球第二大死亡原因，超过癌症相关死亡。传统的帕金森病诊断主要依赖临床经验和主观评估，不但耗时耗力，准确性也受到限制。帕金森病早期诊断困难，传统方法耗时费钱，所以需要寻找更加客观、便捷的诊断手段。近年来，语音分析技术在帕金森病诊断领域展现出巨大潜力，研究人员开展利用人工智能和机器学习技术，通过语音分析诊断早期PD的研究，结果显示模型准确率达91.11%。随着大数据技术的飞速发展，Hadoop和Spark等分布式计算框架为处理海量医疗数据提供了强有力的技术支撑，让我们能够从更深层次挖掘语音数据中蕴含的病理特征信息。 本研究意义，从医疗价值角度来看，系统能够通过分析语音特征指标，准确识别帕金森病患者与健康人群的差异性特征，为医生提供客观的诊断参考依据，有效弥补传统诊断方法主观性强、准确率不高的不足。该系统采用非侵入性的语音采集方式，患者只需进行简单的发声测试就能完成数据采集，大大降低了检查成本和患者负担，特别适合在基层医疗机构推广应用。从技术创新层面来说，项目将Hadoop分布式存储与Spark大数据处理技术应用于医疗诊断领域，实现了对海量语音数据的高效处理和深度挖掘，为医疗大数据分析提供了新的技术路径。系统构建的四维分析框架，涵盖数据整体画像、核心特征差异对比、关联性挖掘以及非线性动力学探索，能够全方位揭示帕金森病的语音病理特征规律。而通过Vue和Echarts的可视化界面设计，让复杂的数据分析结果以直观的图表形式呈现，极大提升了医疗数据的可读性和实用性，为临床决策和科研分析提供了便利工具。
基于Hadoop+Spark的帕金森病数据可视化分析系统-技术选型
大数据框架：Hadoop+Spark（本次没用Hive，支持定制） 开发语言：Python+Java（两个版本都支持） 后端框架：Django+Spring Boot(Spring+SpringMVC+Mybatis)（两个版本都支持） 前端：Vue+ElementUI+Echarts+HTML+CSS+JavaScript+jQuery 详细技术点：Hadoop、HDFS、Spark、Spark SQL、Pandas、NumPy 数据库：MySQL
基于Hadoop+Spark的帕金森病数据可视化分析系统-视频展示
基于Hadoop+Spark的帕金森病数据可视化分析系统-图片展示
基于Hadoop+Spark的帕金森病数据可视化分析系统-代码展示
//大数据部分代码展示
from pyspark.sql import SparkSession
from pyspark.sql.functions import *
from pyspark.ml.feature import VectorAssembler
from pyspark.ml.stat import Correlation
from pyspark.ml.classification import RandomForestClassifier
from pyspark.ml.evaluation import MulticlassClassificationEvaluator
import numpy as np
import pandas as pd
from scipy.stats import ttest_ind
from django.http import JsonResponse
import json
def parkinson_group_profile_analysis(request):
spark = SparkSession.builder.appName("ParkinsonGroupAnalysis").getOrCreate()
df = spark.read.format("jdbc").option("url", "jdbc:mysql://localhost:3306/parkinson_db").option("dbtable", "parkinson_data").option("user", "root").option("password", "password").load()
total_count = df.count()
patient_count = df.filter(col("status") == 1).count()
healthy_count = df.filter(col("status") == 0).count()
balance_ratio = patient_count / healthy_count if healthy_count > 0 else 0
feature_columns = ["MDVP_Fo_Hz", "MDVP_Fhi_Hz", "MDVP_Flo_Hz", "MDVP_Jitter_percent", "MDVP_Jitter_Abs", "MDVP_RAP", "MDVP_PPQ", "Jitter_DDP", "MDVP_Shimmer", "MDVP_Shimmer_dB", "Shimmer_APQ3", "Shimmer_APQ5", "MDVP_APQ", "Shimmer_DDA", "NHR", "HNR", "RPDE", "DFA", "spread1", "spread2", "D2", "PPE"]
overall_stats = df.select([mean(col(c)).alias(f"{c}_mean") for c in feature_columns] + [stddev(col(c)).alias(f"{c}_std") for c in feature_columns] + [min(col(c)).alias(f"{c}_min") for c in feature_columns] + [max(col(c)).alias(f"{c}_max") for c in feature_columns]).collect()[0].asDict()
patient_stats = df.filter(col("status") == 1).select([mean(col(c)).alias(f"{c}_mean") for c in feature_columns] + [stddev(col(c)).alias(f"{c}_std") for c in feature_columns]).collect()[0].asDict()
healthy_stats = df.filter(col("status") == 0).select([mean(col(c)).alias(f"{c}_mean") for c in feature_columns] + [stddev(col(c)).alias(f"{c}_std") for c in feature_columns]).collect()[0].asDict()
variance_comparison = {}
for feature in feature_columns:
patient_variance = patient_stats.get(f"{feature}_std", 0) ** 2
healthy_variance = healthy_stats.get(f"{feature}_std", 0) ** 2
variance_ratio = patient_variance / healthy_variance if healthy_variance > 0 else 0
variance_comparison[feature] = {"patient_variance": patient_variance, "healthy_variance": healthy_variance, "variance_ratio": variance_ratio}
result_data = {"sample_balance": {"total_count": total_count, "patient_count": patient_count, "healthy_count": healthy_count, "balance_ratio": balance_ratio}, "overall_statistics": overall_stats, "patient_profile": patient_stats, "healthy_profile": healthy_stats, "variance_analysis": variance_comparison}
spark.stop()
return JsonResponse({"status": "success", "data": result_data, "message": "患者群体画像分析完成"})
def voice_feature_difference_analysis(request):
spark = SparkSession.builder.appName("VoiceFeatureDifference").getOrCreate()
df = spark.read.format("jdbc").option("url", "jdbc:mysql://localhost:3306/parkinson_db").option("dbtable", "parkinson_data").option("user", "root").option("password", "password").load()
pitch_features = ["MDVP_Fo_Hz", "MDVP_Fhi_Hz", "MDVP_Flo_Hz"]
jitter_features = ["MDVP_Jitter_percent", "MDVP_Jitter_Abs", "MDVP_RAP", "MDVP_PPQ", "Jitter_DDP"]
shimmer_features = ["MDVP_Shimmer", "MDVP_Shimmer_dB", "Shimmer_APQ3", "Shimmer_APQ5", "MDVP_APQ"]
voice_quality_features = ["NHR", "HNR"]
all_features = pitch_features + jitter_features + shimmer_features + voice_quality_features
patient_data = df.filter(col("status") == 1).select(*all_features).toPandas()
healthy_data = df.filter(col("status") == 0).select(*all_features).toPandas()
statistical_results = {}
for feature_group, features in [("pitch", pitch_features), ("jitter", jitter_features), ("shimmer", shimmer_features), ("voice_quality", voice_quality_features)]:
group_results = {}
for feature in features:
patient_values = patient_data[feature].values
healthy_values = healthy_data[feature].values
patient_mean = np.mean(patient_values)
healthy_mean = np.mean(healthy_values)
patient_std = np.std(patient_values)
healthy_std = np.std(healthy_values)
t_stat, p_value = ttest_ind(patient_values, healthy_values)
effect_size = abs(patient_mean - healthy_mean) / np.sqrt((patient_std**2 + healthy_std**2) / 2)
significance_level = "高度显著" if p_value < 0.01 else "显著" if p_value < 0.05 else "不显著"
group_results[feature] = {"patient_mean": patient_mean, "healthy_mean": healthy_mean, "patient_std": patient_std, "healthy_std": healthy_std, "difference": patient_mean - healthy_mean, "t_statistic": t_stat, "p_value": p_value, "effect_size": effect_size, "significance": significance_level}
statistical_results[feature_group] = group_results
feature_ranking = []
for feature_group in statistical_results:
for feature, stats in statistical_results[feature_group].items():
feature_ranking.append({"feature": feature, "group": feature_group, "effect_size": stats["effect_size"], "p_value": stats["p_value"], "significance": stats["significance"]})
feature_ranking.sort(key=lambda x: x["effect_size"], reverse=True)
spark.stop()
return JsonResponse({"status": "success", "data": {"difference_analysis": statistical_results, "feature_importance_ranking": feature_ranking}, "message": "语音特征差异性分析完成"})
def feature_correlation_and_importance_analysis(request):
spark = SparkSession.builder.appName("FeatureCorrelationImportance").getOrCreate()
df = spark.read.format("jdbc").option("url", "jdbc:mysql://localhost:3306/parkinson_db").option("dbtable", "parkinson_data").option("user", "root").option("password", "password").load()
feature_columns = ["MDVP_Fo_Hz", "MDVP_Fhi_Hz", "MDVP_Flo_Hz", "MDVP_Jitter_percent", "MDVP_Jitter_Abs", "MDVP_RAP", "MDVP_PPQ", "Jitter_DDP", "MDVP_Shimmer", "MDVP_Shimmer_dB", "Shimmer_APQ3", "Shimmer_APQ5", "MDVP_APQ", "Shimmer_DDA", "NHR", "HNR", "RPDE", "DFA", "spread1", "spread2", "D2", "PPE"]
assembler = VectorAssembler(inputCols=feature_columns, outputCol="features")
df_assembled = assembler.transform(df)
correlation_matrix = Correlation.corr(df_assembled, "features", "pearson").collect()[0][0]
corr_array = correlation_matrix.toArray()
status_correlations = {}
for i, feature in enumerate(feature_columns):
feature_vector = df.select(feature, "status").rdd.map(lambda row: [float(row[0]), float(row[1])]).collect()
feature_vals = [x[0] for x in feature_vector]
status_vals = [x[1] for x in feature_vector]
correlation_coeff = np.corrcoef(feature_vals, status_vals)[0, 1]
status_correlations[feature] = {"correlation": correlation_coeff, "abs_correlation": abs(correlation_coeff)}
sorted_correlations = sorted(status_correlations.items(), key=lambda x: x[1]["abs_correlation"], reverse=True)
train_df, test_df = df_assembled.randomSplit([0.8, 0.2], seed=42)
rf = RandomForestClassifier(featuresCol="features", labelCol="status", numTrees=100, seed=42)
rf_model = rf.fit(train_df)
feature_importance = rf_model.featureImportances.toArray()
importance_ranking = []
for i, feature in enumerate(feature_columns):
importance_ranking.append({"feature": feature, "importance_score": float(feature_importance[i]), "correlation_with_status": status_correlations[feature]["correlation"]})
importance_ranking.sort(key=lambda x: x["importance_score"], reverse=True)
jitter_features = ["MDVP_Jitter_percent", "MDVP_Jitter_Abs", "MDVP_RAP", "MDVP_PPQ", "Jitter_DDP"]
shimmer_features = ["MDVP_Shimmer", "MDVP_Shimmer_dB", "Shimmer_APQ3", "Shimmer_APQ5", "MDVP_APQ"]
jitter_correlation_matrix = {}
shimmer_correlation_matrix = {}
for i, feat1 in enumerate(jitter_features):
jitter_correlation_matrix[feat1] = {}
for j, feat2 in enumerate(jitter_features):
idx1 = feature_columns.index(feat1)
idx2 = feature_columns.index(feat2)
jitter_correlation_matrix[feat1][feat2] = float(corr_array[idx1][idx2])
for i, feat1 in enumerate(shimmer_features):
shimmer_correlation_matrix[feat1] = {}
for j, feat2 in enumerate(shimmer_features):
idx1 = feature_columns.index(feat1)
idx2 = feature_columns.index(feat2)
shimmer_correlation_matrix[feat1][feat2] = float(corr_array[idx1][idx2])
predictions = rf_model.transform(test_df)
evaluator = MulticlassClassificationEvaluator(labelCol="status", predictionCol="prediction", metricName="accuracy")
model_accuracy = evaluator.evaluate(predictions)
result_data = {"status_correlations": dict(sorted_correlations), "feature_importance_ranking": importance_ranking, "jitter_internal_correlations": jitter_correlation_matrix, "shimmer_internal_correlations": shimmer_correlation_matrix, "model_performance": {"accuracy": model_accuracy, "top_5_features": [item["feature"] for item in importance_ranking[:5]]}}
spark.stop()
基于Hadoop+Spark的帕金森病数据可视化分析系统-结语
💕💕
💟💟如果大家有任何疑虑，欢迎在下方位置详细交流。
