grep日志查询实践：联合查询加排序

程序员非鱼
153 阅读3分钟

最终指令

grep pdms*230530* -e "ERROR" | sort -k 2.1,2.8 | head -n 10

指令解释

指令 grep pdms*230530* -e "ERROR" | sort -k 2.1,2.8 的含义是：

  1. grep pdms*230530* -e "ERROR"：这部分指令使用 grep 命令来搜索包含 pdms、230530 和 ERROR 的行。pdms*230530* 是一个通配符模式，表示匹配以 pdms 开头，中间包含任意字符和 230530，然后跟着任意字符的行。-e "ERROR" 表示搜索包含字符串 ERROR 的行。

  2. pdms*230530*：这是一个通配符模式，用于匹配以 "pdms" 开头，中间包含任意字符和 "230530" 的文件名。它用于联合查询多个文件。

  3. -e "ERROR"：这个部分指定了搜索的模式，即匹配包含字符串 "ERROR" 的行。

  4. head -n 10: 仅显示前一个命名结果的前10行。

  5. |：这个符号是管道操作符，将前一个命令的输出作为后一个命令的输入。

  6. sort -k 2.1,2.8：接收 grep 命令的输出作为输入，并根据第二个字段的排序键对这些行进行排序。-k 2.1,2.8 表示以第二个字段的第一个字符到第八个字符作为排序键。

  7. 排序字段说明：以空格划分，

    1. 如行内容为pdms_47_230530_101153.log:[2023-05-30 10:20:48,187][ERROR]: /app/flexiv_pdms_service/robot/views.py的字段划分如下：
      字段表述字段内容
      第一字段pdms_47_230530_101153.log:[2023-05-30
      第二字段10:20:48,187][ERROR]:
      第三字段/app/flexiv_pdms_service/robot/views.py

示例

  • 优化步骤

    • 第一步查询到有多个日志文件
    • 第二步使用grep联合查询多个文件，并查询文件中包含“ERROR”关键字
    • 第三步将查询结果按照一定规则排序

  • 展示示例

root@pdms_service:/data/pdms/log$ ls | grep 'pdms.*230530.*' | head -n 10
pdms_10_230530_095105.log
pdms_10_230530_095346.log
pdms_10_230530_095451.log
pdms_10_230530_100920.log
pdms_10_230530_101154.log
pdms_10_230530_102301.log
pdms_11_230530_094444.log
pdms_11_230530_095104.log
pdms_11_230530_095344.log
pdms_11_230530_095449.log

root@pdms_service:/data/pdms/log$ grep pdms*230530* -e "ERROR" | head -n 10
pdms_21_230530_094441.log:[2023-05-30 09:47:46,738][ERROR]: process_exception, fvr_common_middleware, 133, /app/flexiv_pdms_service/flexiv_pdms/utils/middleware/fvr_common_middleware.py 
pdms_21_230530_094441.log:[2023-05-30 09:47:46,739][ERROR]: process_exception, fvr_common_middleware, 135, /app/flexiv_pdms_service/flexiv_pdms/utils/middleware/fvr_common_middleware.py 
pdms_21_230530_094441.log:[2023-05-30 09:47:46,739][ERROR]: process_exception, fvr_common_middleware, 162, /app/flexiv_pdms_service/flexiv_pdms/utils/middleware/fvr_common_middleware.py 
pdms_21_230530_094441.log:[2023-05-30 09:49:57,806][ERROR]: process_exception, fvr_common_middleware, 133, /app/flexiv_pdms_service/flexiv_pdms/utils/middleware/fvr_common_middleware.py 
pdms_21_230530_094441.log:[2023-05-30 09:49:57,807][ERROR]: process_exception, fvr_common_middleware, 135, /app/flexiv_pdms_service/flexiv_pdms/utils/middleware/fvr_common_middleware.py 
pdms_21_230530_094441.log:[2023-05-30 09:49:57,808][ERROR]: process_exception, fvr_common_middleware, 162, /app/flexiv_pdms_service/flexiv_pdms/utils/middleware/fvr_common_middleware.py 
pdms_21_230530_095448.log:[2023-05-30 10:01:24,106][ERROR]: process_exception, fvr_common_middleware, 133, /app/flexiv_pdms_service/flexiv_pdms/utils/middleware/fvr_common_middleware.py 
pdms_21_230530_095448.log:[2023-05-30 10:01:24,107][ERROR]: process_exception, fvr_common_middleware, 135, /app/flexiv_pdms_service/flexiv_pdms/utils/middleware/fvr_common_middleware.py 
pdms_21_230530_095448.log:[2023-05-30 10:01:24,107][ERROR]: process_exception, fvr_common_middleware, 145, /app/flexiv_pdms_service/flexiv_pdms/utils/middleware/fvr_common_middleware.py 
pdms_21_230530_095448.log:[2023-05-30 10:01:24,107][ERROR]: process_exception, fvr_common_middleware, 162, /app/flexiv_pdms_service/flexiv_pdms/utils/middleware/fvr_common_middleware.py 

root@pdms_service:/data/pdms/log$ grep pdms*230530* -e "ERROR" | sort -k 2.1,2.8 | head -n 10
pdms_676_230530_073103.log:[2023-05-30 08:33:38,600][ERROR]: process_exception, fvr_common_middleware, 133, /app/flexiv_pdms_service/flexiv_pdms/utils/middleware/fvr_common_middleware.py 
pdms_676_230530_073103.log:[2023-05-30 08:33:38,600][ERROR]: process_exception, fvr_common_middleware, 135, /app/flexiv_pdms_service/flexiv_pdms/utils/middleware/fvr_common_middleware.py 
pdms_676_230530_073103.log:[2023-05-30 08:33:38,601][ERROR]: process_exception, fvr_common_middleware, 162, /app/flexiv_pdms_service/flexiv_pdms/utils/middleware/fvr_common_middleware.py 
pdms_604_230530_055933.log:[2023-05-30 08:39:37,344][ERROR]: process_exception, fvr_common_middleware, 133, /app/flexiv_pdms_service/flexiv_pdms/utils/middleware/fvr_common_middleware.py 
pdms_604_230530_055933.log:[2023-05-30 08:39:37,344][ERROR]: process_exception, fvr_common_middleware, 135, /app/flexiv_pdms_service/flexiv_pdms/utils/middleware/fvr_common_middleware.py 
pdms_604_230530_055933.log:[2023-05-30 08:39:37,345][ERROR]: process_exception, fvr_common_middleware, 162, /app/flexiv_pdms_service/flexiv_pdms/utils/middleware/fvr_common_middleware.py 
pdms_51_230530_094245.log:[2023-05-30 09:44:01,813][ERROR]: process_exception, fvr_common_middleware, 133, /app/flexiv_pdms_service/flexiv_pdms/utils/middleware/fvr_common_middleware.py 
pdms_51_230530_094245.log:[2023-05-30 09:44:01,814][ERROR]: process_exception, fvr_common_middleware, 135, /app/flexiv_pdms_service/flexiv_pdms/utils/middleware/fvr_common_middleware.py 
pdms_51_230530_094245.log:[2023-05-30 09:44:01,814][ERROR]: process_exception, fvr_common_middleware, 162, /app/flexiv_pdms_service/flexiv_pdms/utils/middleware/fvr_common_middleware.py 
pdms_37_230530_094442.log:[2023-05-30 09:46:55,539][ERROR]: process_exception, fvr_common_middleware, 133, /app/flexiv_pdms_service/flexiv_pdms/utils/middleware/fvr_common_middleware.py

异常

  • 问题：“Binary file file_name matches”，表示grep无法识别二进制文件
  • 解决方式：添加-a参数，将二进制文件视为文本文件
root@defensor-gadgets:~/app/flexiv_portal_service$ grep log/worker_* -e "save_so error"
Binary file log/worker_default.log matches
root@defensor-gadgets:~/app/flexiv_portal_service$ grep -a log/worker_* -e "save_so error"
...
avatar
文章
阅读
粉丝