海洋塑料污染数据分析与可视化系统 - 简介
海洋环境数据如何用Hadoop处理？这套污染分析可视化系统告诉你答案 大数据毕设技术栈还停留在基础阶段？Hadoop+Spark海洋数据分析提升竞争力
海洋塑料污染数据分析与可视化系统系统 -技术
开发语言：java或Python
数据库：MySQL
系统架构：B/S
前端：Vue+ElementUI+HTML+CSS+JavaScript+jQuery+Echarts
大数据框架：Hadoop+Spark（本次没用Hive，支持定制）
后端框架：Django+Spring Boot(Spring+SpringMVC+Mybatis)
海洋塑料污染数据分析与可视化系统 - 背景
海洋塑料污染已成为当今全球面临的重大环境挑战之一，随着工业化进程的不断推进和塑料制品使用量的急剧增长，大量塑料废弃物通过河流、直接倾倒等途径进入海洋生态系统，对海洋生物多样性和生态平衡造成严重威胁。传统的海洋污染监测手段往往局限于小范围取样和简单统计分析，难以应对日益庞大的监测数据量和复杂的分析需求。现有的数据处理方式多采用单机处理模式，在面对跨年度、多区域、多类型的海洋污染数据时存在处理效率低下、分析维度单一等问题。与此同时，环境科学研究对数据可视化的需求日益增长，研究人员迫切需要能够直观展现污染趋势、分布规律和关联关系的分析工具。大数据技术的快速发展为解决这些问题提供了新的技术路径，通过分布式计算框架可以高效处理海量环境监测数据，结合现代可视化技术能够更好地支撑科学研究和决策制定。
本课题的研究具有一定的理论价值和实践意义，虽然作为毕业设计项目在规模和深度上有所限制，但仍能在多个方面发挥积极作用。从技术层面来看，该系统将大数据处理技术与环境科学研究相结合，探索了Hadoop+Spark框架在环境数据分析中的应用可能性，为类似的环境监测项目提供了技术参考和实现思路。在实际应用方面，系统能够协助环境研究人员更高效地处理和分析海洋污染数据，通过多维度的数据挖掘和直观的可视化展示，有助于发现污染规律和识别重点治理区域，为环境保护工作提供一定的数据支撑。教育意义方面，该项目将理论知识与实际环境问题相结合，体现了计算机技术服务社会的应用价值，有助于培养学生的综合实践能力和社会责任感。社会层面而言，虽然作为学术项目其直接影响有限，但通过技术手段关注环境问题，体现了新一代技术人员对环境保护的重视和参与意识，具有一定的示范效应和教育意义。
海洋塑料污染数据分析与可视化系统 -视频展示
海洋塑料污染数据分析与可视化系统 -图片展示
海洋塑料污染数据分析与可视化系统 -代码展示
from pyspark.sql.functions import col, year, month, when, sum as spark_sum, avg, count, desc
from pyspark.ml.clustering import KMeans
from pyspark.ml.feature import VectorAssembler
import pandas as pd
from django.http import JsonResponse
from datetime import datetime
spark = SparkSession.builder.appName("OceanPlasticAnalysis").config("spark.sql.adaptive.enabled", "true").config("spark.sql.adaptive.coalescePartitions.enabled", "true").getOrCreate()
def analyze_temporal_pollution_trends(request):
df = spark.read.format("jdbc").option("url", "jdbc:mysql://localhost:3306/ocean_db").option("dbtable", "pollution_data").option("user", "root").option("password", "password").load()
df_with_year = df.withColumn("year", year(col("date")))
df_with_month = df.withColumn("month", month(col("date")))
yearly_trends = df_with_year.groupBy("year").agg(spark_sum("plastic_weight_kg").alias("total_weight")).orderBy("year")
yearly_pandas = yearly_trends.toPandas()
monthly_trends = df_with_month.groupBy("month").agg(spark_sum("plastic_weight_kg").alias("total_weight"), avg("plastic_weight_kg").alias("avg_weight")).orderBy("month")
monthly_pandas = monthly_trends.toPandas()
seasonal_df = df.withColumn("season", when(month(col("date")).isin([12, 1, 2]), "Winter").when(month(col("date")).isin([3, 4, 5]), "Spring").when(month(col("date")).isin([6, 7, 8]), "Summer").otherwise("Autumn"))
seasonal_trends = seasonal_df.groupBy("season").agg(spark_sum("plastic_weight_kg").alias("total_weight")).orderBy("total_weight", ascending=False)
seasonal_pandas = seasonal_trends.toPandas()
plastic_yearly_trends = df_with_year.groupBy("year", "plastic_type").agg(spark_sum("plastic_weight_kg").alias("weight")).orderBy("year", "plastic_type")
plastic_yearly_pandas = plastic_yearly_trends.toPandas()
result_data = {"yearly_data": yearly_pandas.to_dict("records"), "monthly_data": monthly_pandas.to_dict("records"), "seasonal_data": seasonal_pandas.to_dict("records"), "plastic_yearly_data": plastic_yearly_pandas.to_dict("records")}
return JsonResponse(result_data)
def analyze_spatial_pollution_distribution(request):
df = spark.read.format("jdbc").option("url", "jdbc:mysql://localhost:3306/ocean_db").option("dbtable", "pollution_data").option("user", "root").option("password", "password").load()
regional_pollution = df.groupBy("region").agg(spark_sum("plastic_weight_kg").alias("total_weight"), count("*").alias("event_count"), avg("plastic_weight_kg").alias("avg_weight")).orderBy("total_weight", ascending=False)
regional_pandas = regional_pollution.toPandas()
regional_plastic_composition = df.groupBy("region", "plastic_type").agg(spark_sum("plastic_weight_kg").alias("weight")).orderBy("region", "weight", ascending=False)
regional_plastic_pandas = regional_plastic_composition.toPandas()
heatmap_data = df.select("latitude", "longitude", "plastic_weight_kg").filter(col("plastic_weight_kg") > 0)
heatmap_pandas = heatmap_data.toPandas()
lat_bins = [-90 + i * 10 for i in range(19)]
lon_bins = [-180 + i * 10 for i in range(37)]
density_df = df.withColumn("lat_bin", ((col("latitude") + 90) / 10).cast("int")).withColumn("lon_bin", ((col("longitude") + 180) / 10).cast("int"))
density_analysis = density_df.groupBy("lat_bin", "lon_bin").agg(count("*").alias("event_density"), spark_sum("plastic_weight_kg").alias("weight_density")).filter(col("event_density") > 0)
density_pandas = density_analysis.toPandas()
top_pollution_spots = df.orderBy(col("plastic_weight_kg").desc()).limit(100)
top_spots_composition = top_pollution_spots.groupBy("plastic_type").agg(spark_sum("plastic_weight_kg").alias("weight"), count("*").alias("count")).orderBy("weight", ascending=False)
top_spots_pandas = top_spots_composition.toPandas()
result_data = {"regional_data": regional_pandas.to_dict("records"), "composition_data": regional_plastic_pandas.to_dict("records"), "heatmap_data": heatmap_pandas.to_dict("records"), "density_data": density_pandas.to_dict("records"), "hotspot_composition": top_spots_pandas.to_dict("records")}
return JsonResponse(result_data)
def analyze_plastic_type_characteristics(request):
df = spark.read.format("jdbc").option("url", "jdbc:mysql://localhost:3306/ocean_db").option("dbtable", "pollution_data").option("user", "root").option("password", "password").load()
plastic_contribution = df.groupBy("plastic_type").agg(spark_sum("plastic_weight_kg").alias("total_weight"), count("*").alias("occurrence_count"), avg("plastic_weight_kg").alias("avg_weight")).orderBy("total_weight", ascending=False)
plastic_pandas = plastic_contribution.toPandas()
total_weight = df.agg(spark_sum("plastic_weight_kg").alias("global_total")).collect()[0]["global_total"]
plastic_percentage = plastic_pandas.copy()
plastic_percentage["percentage"] = (plastic_percentage["total_weight"] / total_weight * 100).round(2)
depth_analysis = df.groupBy("plastic_type").agg(avg("depth_meters").alias("avg_depth"), spark_sum("plastic_weight_kg").alias("weight")).orderBy("avg_depth")
depth_pandas = depth_analysis.toPandas()
regional_distribution = df.groupBy("plastic_type", "region").agg(spark_sum("plastic_weight_kg").alias("weight")).orderBy("plastic_type", "weight", ascending=False)
regional_dist_pandas = regional_distribution.toPandas()
plastic_region_percentage = regional_dist_pandas.groupby("plastic_type").apply(lambda x: x.assign(region_percentage=(x["weight"] / x["weight"].sum() * 100).round(2))).reset_index(drop=True)
assembler = VectorAssembler(inputCols=["latitude", "longitude"], outputCol="location_features")
location_df = assembler.transform(df.select("plastic_type", "latitude", "longitude", "plastic_weight_kg"))
kmeans = KMeans(k=8, featuresCol="location_features", predictionCol="cluster")
model = kmeans.fit(location_df)
clustered_df = model.transform(location_df)
cluster_analysis = clustered_df.groupBy("cluster", "plastic_type").agg(spark_sum("plastic_weight_kg").alias("cluster_weight"), count("*").alias("cluster_count")).orderBy("cluster", "cluster_weight", ascending=False)
cluster_pandas = cluster_analysis.toPandas()
result_data = {"plastic_contribution": plastic_percentage.to_dict("records"), "depth_analysis": depth_pandas.to_dict("records"), "regional_distribution": plastic_region_percentage.to_dict("records"), "cluster_analysis": cluster_pandas.to_dict("records")}
return JsonResponse(result_data)
海洋塑料污染数据分析与可视化系统 -结语
海洋环境数据如何用Hadoop处理？这套污染分析可视化系统告诉你答案
大数据毕设技术栈还停留在基础阶段？Hadoop+Spark海洋数据分析提升竞争力
如果你觉得内容不错，欢迎一键三连（点赞、收藏、关注）支持一下！也欢迎在评论区或在博客主页上私信联系留下你的想法或提出宝贵意见，期待与大家交流探讨！谢谢！
⚡⚡如果遇到具体的技术问题或计算机毕设方面需求！你也可以在个人主页上咨询我~~
