【大数据】Neo4j图数据查询python公共类

持续创作，加速成长！这是我参与「掘金日新计划 · 10 月更文挑战」的第21天，点击查看活动详情

前言

昨天的文章中，我们使用neo4j的cypher查询语句中的match关键字，对图数据库中的节点和关系进行了简单的查询操作，主要包括单节点查询、单节点情况下的属性查询、Id查询；多节点查询、多节点情况下的关系查询，以及变长路径查询，而这些cypher语句能够在neo4j的可视化界面中进行查询操作，而通过后端进行查询的话，还需要依靠py2neo这个类库，因此今天的文章记录写几个python的公共类，用于节点的查询操作，并把数据封装为前端所需的形式。

---

数据库连接

class connect2Neo4J():
    def __init__(self):
        self.userName = "neo4j"
        self.password = "{{password}}"
        self.neo4jUrl = 'http://127.0.0.1:17474'
        self.conn = Graph(self.neo4jUrl, auth=(self.userName, self.password))

其中数据库连接还是使用之前写的方法，利用py2neo中的Graph类进行数据库连接

查询方法

node_matcher = NodeMatcher(self.conn)
data = node_matcher.match(labels)

我们之前使用了NodeMatcher和NodeMatch方法查询了数据库，但是对于复杂的查询语句如多节点和关系之间的查询，仅用这两个NodeMatcher和NodeMatch方法就显得不够用了，因此针对我们的需求，设计如下方法：

查询全部节点

# 查询全部节点
def getAllNodes(self, labels = None, properties = None, limit = None):
    cypher = "match (n) return n"
    if labels:
        cypher = f"match (n:{labels}) return id(n),n"
    if properties:
        cypher = "match (n:{0}{{name:'{1}'}}) return id(n),n".format(labels,properties)
    if limit:
        cypher += ' limit {}'.format(limit)
    else:
        cypher = f"match (n) return n limit {self.maxNodes}"
    print(cypher)
    result = self.conn.run(cypher)
    return result.data()

其中参数为labels、properties、limit，当都为空的之后返回参数self.maxNodes大小的节点， 如果不为空，则根据labels，properties，limit对应参数进行值返回：

测试脚本如下：

search4Node().getAllNodes(labels="company",limit=20)

测试结果如下：

match (n:company) return id(n),n limit 20
{'id(n)': 6214, 'n': Node('company', name='haskell')}
{'id(n)': 6215, 'n': Node('company', name='brainvire')}
{'id(n)': 6220, 'n': Node('company', name='slack_morphism_project')}
{'id(n)': 6222, 'n': Node('company', name='misp-project')}
{'id(n)': 6227, 'n': Node('company', name='solarwinds')}
{'id(n)': 6238, 'n': Node('company', name='shortpixel')}
{'id(n)': 6242, 'n': Node('company', name='crealogix')}
{'id(n)': 6244, 'n': Node('company', name='linuxmint')}
{'id(n)': 6254, 'n': Node('company', name='premium-themes')}
{'id(n)': 6255, 'n': Node('company', name='resmush.it')}
{'id(n)': 6260, 'n': Node('company', name='wpwhitesecurity')}
{'id(n)': 6262, 'n': Node('company', name='ikus-soft')}
{'id(n)': 6263, 'n': Node('company', name='d-bus_project')}
{'id(n)': 6264, 'n': Node('company', name='linux')}
{'id(n)': 6265, 'n': Node('company', name='web-based_student_clearance_system_project')}
{'id(n)': 6274, 'n': Node('company', name='trendmicro')}
{'id(n)': 6276, 'n': Node('company', name='metaslider')}
{'id(n)': 6278, 'n': Node('company', name='cert')}
{'id(n)': 6292, 'n': Node('company', name='wpdarko')}
{'id(n)': 6302, 'n': Node('company', name='panini')}

测试脚本2如下：

search4Node().getAllNodes(labels="company",properties= "haskell",limit=20)

测试结果如下：

match (n:company{name:'haskell'}) return id(n),n limit 20

{'id(n)': 6214, 'n': Node('company', name='haskell')}

查询两个节点的连接信息

def get2PointRel(self, nodeA, nodeB, propertiesA=None, propertiesB=None, limit=10):
    # match (n:company{name:'microsoft'}) -[r]-> (m:product) return m,r,n
    cypher = f"match (n:{nodeA}) -[r]-> (m:{nodeB})  return id(m),m ,id(r),r,id(n),n"
    if propertiesA is not None and propertiesB is None:
        cypher = f"match (n:{nodeA}{{name:'{propertiesA}'}}) -[r]-> (m:{nodeB})  return id(m),m ,id(r),r,id(n),n"
    if propertiesB and propertiesA is None:
        cypher = f"match (n:{nodeA}) -[r]-> (m:{nodeB}{{name:'{propertiesB}'}})  return id(m),m ,id(r),r,id(n),n"
    if propertiesA is not None and propertiesB is not None:
        cypher = f"match (n:{nodeA}{{name:'{propertiesA}'}}) -[r]-> (m:{nodeB}{{name:'{propertiesB}'}}) return id(m),m ,id(r),r,id(n),n"
    if limit:
        cypher += ' limit {}'.format(limit)
    print(cypher)
    result = self.conn.run(cypher)
    return result.data()

根据cypher语句，查询关系语句必定包含两个节点，因此nodeA和nodeB两个节点是必传参数，而节点属性则是选传的参数，分别为属性A和属性B，limit限制节点返回数量，避免数据量过大时造成查询速度缓慢，方法方面，则根据传参的不同生成不同的cypher语句，因此也能够返回不同的节点关系等。

测试用例1

search4Node().get2PointRel("company","product","microsoft","office",None)

测试结果1

{'id(m)': 6635, 'm': Node('product', name='office'), 'id(r)': 5672, 'r': 拥有(Node('company', name='microsoft'), Node('product', name='office')), 'id(n)': 6379, 'n': Node('company', name='microsoft')}

测试用例2

search4Node().get2PointRel("company","product")

测试结果2

{'id(m)': 6273, 'm': Node('product', name='aeson'), 'id(r)': 5259, 'r': 拥有(Node('company', name='haskell'), Node('product', name='aeson')), 'id(n)': 6214, 'n': Node('company', name='haskell')}
{'id(m)': 6413, 'm': Node('product', name='disable_user_login'), 'id(r)': 5242, 'r': 拥有(Node('company', name='brainvire'), Node('product', name='disable_user_login')), 'id(n)': 6215, 'n': Node('company', name='brainvire')}
{'id(m)': 6295, 'm': Node('product', name='slack_morphism'), 'id(r)': 5244, 'r': 拥有(Node('company', name='slack_morphism_project'), Node('product', name='slack_morphism')), 'id(n)': 6220, 'n': Node('company', name='slack_morphism_project')}
{'id(m)': 6245, 'm': Node('product', name='malware_information_sharing_platform'), 'id(r)': 5304, 'r': 拥有(Node('company', name='misp-project'), Node('product', name='malware_information_sharing_platform')), 'id(n)': 6222, 'n': Node('company', name='misp-project')}
{'id(m)': 6714, 'm': Node('product', name='network_configuration_manager'), 'id(r)': 5670, 'r': 拥有(Node('company', name='solarwinds'), Node('product', name='network_configuration_manager')), 'id(n)': 6227, 'n': Node('company', name='solarwinds')}
{'id(m)': 6256, 'm': Node('product', name='enable_media_replace'), 'id(r)': 5262, 'r': 拥有(Node('company', name='shortpixel'), Node('product', name='enable_media_replace')), 'id(n)': 6238, 'n': Node('company', name='shortpixel')}
{'id(m)': 6296, 'm': Node('product', name='ebics_server'), 'id(r)': 5302, 'r': 拥有(Node('company', name='crealogix'), Node('product', name='ebics_server')), 'id(n)': 6242, 'n': Node('company', name='crealogix')}
{'id(m)': 6706, 'm': Node('product', name='warpinator'), 'id(r)': 5247, 'r': 拥有(Node('company', name='linuxmint'), Node('product', name='warpinator')), 'id(n)': 6244, 'n': Node('company', name='linuxmint')}
{'id(m)': 6236, 'm': Node('product', name='cryptocurrency_pricing_list_and_ticker'), 'id(r)': 5241, 'r': 拥有(Node('company', name='premium-themes'), Node('product', name='cryptocurrency_pricing_list_and_ticker')), 'id(n)': 6254, 'n': Node('company', name='premium-themes')}
{'id(m)': 6216, 'm': Node('product', name='resmush.it_image_optimizer'), 'id(r)': 5243, 'r': 拥有(Node('company', name='resmush.it'), Node('product', name='resmush.it_image_optimizer')), 'id(n)': 6255, 'n': Node('company', name='resmush.it')}

因此通过不同参数，能够获得不同的返回结果。

查询仨个节点的连接信息

同理，本案例使用节点最多三跳，因此还需要一个查询3个节点关系的接口，或者使用公司+产品获得一层关系，再用产品+cve编号查询获取第二->第三跳信息，但是这样感觉查询三节点和关系效率较低，因此还是封一个方法吧。

又臭又长的代码，之后学明白了再看咋优化吧。。。

def get3PointRel(self, nodeA, nodeB, nodeC, propertiesA=None, propertiesB=None, propertiesC=None, limit=10):
    # match (n:company{name:'microsoft'}) -[r]-> (m:product) - [t]->(o:cvenumberNode) return m,r,n,t,o
    #cypher = f"match (n:{nodeA}) -[r]-> (m:{nodeB})   return id(m),m ,id(r),r,id(n),n"
    cypher = ""
    if propertiesA is not None and propertiesB is None and propertiesC is None:
        cypher = f"match (n:{nodeA}{{name:'{propertiesA}'}}) -[r]-> (m:{nodeB}) - [t]->(o:{nodeC}) return id(n) ,n ,id(m),m, id (o), o ,id(r) ,r ,id(t), t"
    if propertiesA is None and propertiesB is not None and propertiesC is None:
        cypher = f"match (n:{nodeA}) -[r]-> (m:{nodeB}{{name:'{propertiesB}'}}) - [t]->(o:{nodeC}) return id(n) ,n ,id(m),m, id (o), o ,id(r) ,r ,id(t), t"
    if propertiesA is  None and propertiesB is None and propertiesC is not None:
        cypher = f"match (n:{nodeA}) -[r]-> (m:{nodeB}) - [t]->(o:{nodeC}{{name:'{propertiesC}'}}) return id(n) ,n ,id(m),m, id (o), o ,id(r) ,r ,id(t), t"


    if propertiesA is not None and propertiesB is not None and propertiesC is None:
        cypher = f"match (n:{nodeA}{{name:'{propertiesA}'}}) -[r]-> (m:{nodeB}{{name:'{propertiesB}'}}) - [t]->(o:{nodeC}) return id(n) ,n ,id(m),m, id (o), o ,id(r) ,r ,id(t), t"
    if propertiesA is not None and propertiesB is not None and propertiesC is  None:
        cypher = f"match (n:{nodeA}) -[r]-> (m:{nodeB}{{name:'{propertiesB}'}}) - [t]->(o:{nodeC}) return id(n) ,n ,id(m),m, id (o), o ,id(r) ,r ,id(t), t"
    if propertiesA is None and propertiesB is not None and propertiesC is not None:
        cypher = f"match (n:{nodeA}) -[r]-> (m:{nodeB}{{name:'{propertiesB}'}}) - [t]->(o:{nodeC}{{name:'{propertiesC}'}}) return id(n) ,n ,id(m),m, id (o), o ,id(r) ,r ,id(t), t"

    if propertiesA is not None and propertiesB is not None and propertiesC is not None:
        cypher = f"match (n:{nodeA}{{name:'{propertiesA}'}}) -[r]-> (m:{nodeB}{{name:'{propertiesB}'}}) - [t]->(o:{nodeC}{{name:'{propertiesC}'}}) return id(n) ,n ,id(m),m, id (o), o ,id(r) ,r ,id(t), t"

    if propertiesA is None and propertiesB is None and propertiesC is None:
        cypher = f"match (n:{nodeA}) -[r]-> (m:{nodeB}) - [t]->(o:{nodeC}) return id(n) ,n ,id(m),m, id (o), o ,id(r) ,r ,id(t), t"
    if limit:
        cypher += ' limit {}'.format(limit)
    print(cypher)
    result = self.conn.run(cypher)
    return result.data()

测试

search4Node().get3PointRel("company", "product", "cvenumberNode")

结果

{'id(n)': 6214, 'n': Node('company', name='haskell'), 'id(m)': 6273, 'm': Node('product', name='aeson'), 'id (o)': 6234, 'o': Node('cvenumberNode', name='CVE-2022-3433'), 'id(r)': 5259, 'r': 拥有(Node('company', name='haskell'), Node('product', name='aeson')), 'id(t)': 5297, 't': 暴露漏洞(Node('product', name='aeson'), Node('cvenumberNode', name='CVE-2022-3433'))}
{'id(n)': 6215, 'n': Node('company', name='brainvire'), 'id(m)': 6413, 'm': Node('product', name='disable_user_login'), 'id (o)': 6237, 'o': Node('cvenumberNode', name='CVE-2022-2350'), 'id(r)': 5242, 'r': 拥有(Node('company', name='brainvire'), Node('product', name='disable_user_login')), 'id(t)': 5299, 't': 暴露漏洞(Node('product', name='disable_user_login'), Node('cvenumberNode', name='CVE-2022-2350'))}
{'id(n)': 6220, 'n': Node('company', name='slack_morphism_project'), 'id(m)': 6295, 'm': Node('product', name='slack_morphism'), 'id (o)': 6657, 'o': Node('cvenumberNode', name='CVE-2022-39292'), 'id(r)': 5244, 'r': 拥有(Node('company', name='slack_morphism_project'), Node('product', name='slack_morphism')), 'id(t)': 5267, 't': 暴露漏洞(Node('product', name='slack_morphism'), Node('cvenumberNode', name='CVE-2022-39292'))}
{'id(n)': 6222, 'n': Node('company', name='misp-project'), 'id(m)': 6245, 'm': Node('product', name='malware_information_sharing_platform'), 'id (o)': 6298, 'o': Node('cvenumberNode', name='CVE-2022-42724'), 'id(r)': 5304, 'r': 拥有(Node('company', name='misp-project'), Node('product', name='malware_information_sharing_platform')), 'id(t)': 5248, 't': 暴露漏洞(Node('product', name='malware_information_sharing_platform'), Node('cvenumberNode', name='CVE-2022-42724'))}
{'id(n)': 6227, 'n': Node('company', name='solarwinds'), 'id(m)': 6714, 'm': Node('product', name='network_configuration_manager'), 'id (o)': 6407, 'o': Node('cvenumberNode', name='CVE-2021-35226'), 'id(r)': 5670, 'r': 拥有(Node('company', name='solarwinds'), Node('product', name='network_configuration_manager')), 'id(t)': 5653, 't': 暴露漏洞(Node('product', name='network_configuration_manager'), Node('cvenumberNode', name='CVE-2021-35226'))}
{'id(n)': 6238, 'n': Node('company', name='shortpixel'), 'id(m)': 6256, 'm': Node('product', name='enable_media_replace'), 'id (o)': 6656, 'o': Node('cvenumberNode', name='CVE-2022-2554'), 'id(r)': 5262, 'r': 拥有(Node('company', name='shortpixel'), Node('product', name='enable_media_replace')), 'id(t)': 5279, 't': 暴露漏洞(Node('product', name='enable_media_replace'), Node('cvenumberNode', name='CVE-2022-2554'))}
{'id(n)': 6242, 'n': Node('company', name='crealogix'), 'id(m)': 6296, 'm': Node('product', name='ebics_server'), 'id (o)': 6279, 'o': Node('cvenumberNode', name='CVE-2022-3442'), 'id(r)': 5302, 'r': 拥有(Node('company', name='crealogix'), Node('product', name='ebics_server')), 'id(t)': 5245, 't': 暴露漏洞(Node('product', name='ebics_server'), Node('cvenumberNode', name='CVE-2022-3442'))}
{'id(n)': 6244, 'n': Node('company', name='linuxmint'), 'id(m)': 6706, 'm': Node('product', name='warpinator'), 'id (o)': 6221, 'o': Node('cvenumberNode', name='CVE-2022-42725'), 'id(r)': 5247, 'r': 拥有(Node('company', name='linuxmint'), Node('product', name='warpinator')), 'id(t)': 5226, 't': 暴露漏洞(Node('product', name='warpinator'), Node('cvenumberNode', name='CVE-2022-42725'))}
{'id(n)': 6254, 'n': Node('company', name='premium-themes'), 'id(m)': 6236, 'm': Node('product', name='cryptocurrency_pricing_list_and_ticker'), 'id (o)': 6412, 'o': Node('cvenumberNode', name='CVE-2021-25044'), 'id(r)': 5241, 'r': 拥有(Node('company', name='premium-themes'), Node('product', name='cryptocurrency_pricing_list_and_ticker')), 'id(t)': 5261, 't': 暴露漏洞(Node('product', name='cryptocurrency_pricing_list_and_ticker'), Node('cvenumberNode', name='CVE-2021-25044'))}
{'id(n)': 6255, 'n': Node('company', name='resmush.it'), 'id(m)': 6216, 'm': Node('product', name='resmush.it_image_optimizer'), 'id (o)': 6291, 'o': Node('cvenumberNode', name='CVE-2022-2448'), 'id(r)': 5243, 'r': 拥有(Node('company', name='resmush.it'), Node('product', name='resmush.it_image_optimizer')), 'id(t)': 5222, 't': 暴露漏洞(Node('product', name='resmush.it_image_optimizer'), Node('cvenumberNode', name='CVE-2022-2448'))}

查询带有公司属性的

search4Node().get3PointRel("company", "product", "cvenumberNode","facebook")

结果：

{'id(n)': 6713, 'n': Node('company', name='facebook'), 'id(m)': 6476, 'm': Node('product', name='hermes'), 'id (o)': 6659, 'o': Node('cvenumberNode', name='CVE-2022-27810'), 'id(r)': 5652, 'r': 拥有(Node('company', name='facebook'), Node('product', name='hermes')), 'id(t)': 5862, 't': 暴露漏洞(Node('product', name='hermes'), Node('cvenumberNode', name='CVE-2022-27810'))}
{'id(n)': 6713, 'n': Node('company', name='facebook'), 'id(m)': 6476, 'm': Node('product', name='hermes'), 'id (o)': 6634, 'o': Node('cvenumberNode', name='CVE-2022-32234'), 'id(r)': 5652, 'r': 拥有(Node('company', name='facebook'), Node('product', name='hermes')), 'id(t)': 5621, 't': 暴露漏洞(Node('product', name='hermes'), Node('cvenumberNode', name='CVE-2022-32234'))}
{'id(n)': 6713, 'n': Node('company', name='facebook'), 'id(m)': 6476, 'm': Node('product', name='hermes'), 'id (o)': 6406, 'o': Node('cvenumberNode', name='CVE-2022-40138'), 'id(r)': 5652, 'r': 拥有(Node('company', name='facebook'), Node('product', name='hermes')), 'id(t)': 5637, 't': 暴露漏洞(Node('product', name='hermes'), Node('cvenumberNode', name='CVE-2022-40138'))}
{'id(n)': 6713, 'n': Node('company', name='facebook'), 'id(m)': 6476, 'm': Node('product', name='hermes'), 'id (o)': 6454, 'o': Node('cvenumberNode', name='CVE-2022-35289'), 'id(r)': 5652, 'r': 拥有(Node('company', name='facebook'), Node('product', name='hermes')), 'id(t)': 5669, 't': 暴露漏洞(Node('product', name='hermes'), Node('cvenumberNode', name='CVE-2022-35289'))}

基本完成了所需的查询功能，节点，双节点关系，三节点关系，剩下就是将代码封装成接口供查询调用。Thanks♪(･ω･)ﾉ~。