数据流

2

回答

易观方舟Argo数据流开启不了，各项服务都正常，怎么办？

朱楠@Argo运营 5个月前

数据流,无法开启 284人浏览 0人关注复制链接

用户15814944725个月前

tail -300f /opt/soft/log/ark-streaming/${appkey}/event0stream.log

2020-02-12 11:30:47.856 [RunnerStream-7eeefe9dbc34306e_event0] INFO  cn.analysys.ark.streaming.scheduler.main.f -start execute newStream：/ark/streaming/schedule/ark1.analysys.xyz/waitrun7eeefe9dbc34306e_event_0_20200212104139
2020-02-12 11:30:47.857 [RunnerStream-7eeefe9dbc34306e_event0] INFO  cn.analysys.ark.streaming.common.zk.a -节点已经存在：/ark/streaming/lock/schedule/7eeefe9dbc34306e_event_0_20200212104139
2020-02-12 11:30:47.857 [RunnerStream-7eeefe9dbc34306e_event0] INFO  c.a.ark.streaming.common.zk.ArkStreamingZKClient -创建锁的类型为=>newStream,创建节点为=>/ark/streaming/lock/schedule/7eeefe9dbc34306e_event_0_20200212104139
2020-02-12 11:30:47.943 [RunnerStream-7eeefe9dbc34306e_event0] INFO  cn.analysys.ark.streaming.scheduler.main.f -Lock success
2020-02-12 11:30:47.945 [RunnerStream-7eeefe9dbc34306e_event0] INFO  cn.analysys.ark.streaming.common.zk.a -准备开始创建节点/ark/streaming/schedule/ark1.analysys.xyz/running/7eeefe9dbc34306e_event_0_20200212104139
2020-02-12 11:30:47.947 [RunnerStream-7eeefe9dbc34306e_event0] INFO  cn.analysys.ark.streaming.common.zk.a -完成添加节点:/ark/streaming/schedule/ark1.analysys.xyz/running/7eeefe9dbc34306e_event_0_20200212104139
2020-02-12 11:30:47.950 [RunnerStream-7eeefe9dbc34306e_event0] INFO  cn.analysys.ark.streaming.common.zk.a -删除节点:/ark/streaming/schedule/ark1.analysys.xyz/waitrun/7eeefe9dbc34306e_event_0_20200212104139
2020-02-12 11:30:47.950 [RunnerStream-7eeefe9dbc34306e_event0] INFO  cn.analysys.ark.streaming.common.zk.a -删除zk节点[/ark/streaming/schedule/ark1.analysys.xyz/waitrun/7eeefe9dbc34306e_event_0_20200212104139]成功
2020-02-12 11:30:47.952 [RunnerStream-7eeefe9dbc34306e_event0] INFO  cn.analysys.ark.streaming.scheduler.main.f -Release lock success
2020-02-12 11:30:47.952 [RunnerStream-7eeefe9dbc34306e_event0] INFO  cn.analysys.ark.streaming.scheduler.main.f -start handle business
2020-02-12 11:30:48.265 [RunnerStream-7eeefe9dbc34306e_event0] INFO  cn.analysys.ark.streaming.common.zk.a -节点已经存在：/ark/streaming/stop/7eeefe9dbc34306e
2020-02-12 11:30:48.265 [RunnerStream-7eeefe9dbc34306e_event0] INFO  cn.analysys.ark.streaming.common.zk.a -删除zk节点[/ark/streaming/schedule/ark1.analysys.xyz/waitrun/7eeefe9dbc34306e_event_0_20200212104139]成功
2020-02-12 11:30:48.268 [RunnerStream-7eeefe9dbc34306e_event0] INFO  cn.analysys.ark.streaming.common.zk.a -删除节点:/ark/streaming/schedule/ark1.analysys.xyz/running/7eeefe9dbc34306e_event_0_20200212104139
2020-02-12 11:30:48.268 [RunnerStream-7eeefe9dbc34306e_event0] INFO  cn.analysys.ark.streaming.common.zk.a -删除zk节点[/ark/streaming/schedule/ark1.analysys.xyz/running/7eeefe9dbc34306e_event_0_20200212104139]成功
2020-02-12 11:30:48.269 [RunnerStream-7eeefe9dbc34306e_event0] ERROR cn.analysys.ark.streaming.scheduler.main.f -java.lang.NullPointerException
	at cn.analysys.ark.streaming.common.redis.ToolRedis.getValueByKey(ToolRedis.java:177)
	at cn.analysys.ark.streaming.service.b.a(ArkServiceEventImpl.java:168)
	at cn.analysys.ark.streaming.scheduler.main.f.a(StreamingRunningRunner.java:121)
	at cn.analysys.ark.streaming.scheduler.main.f.run(StreamingRunningRunner.java:69)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

2020-02-12 11:30:48.270 [RunnerStream-7eeefe9dbc34306e_event0] INFO  cn.analysys.ark.streaming.service.f -释放锁:/ark/streaming/lock/metadata/db_7eeefe9dbc34306e/event_write
2020-02-12 11:30:48.270 [RunnerStream-7eeefe9dbc34306e_event0] INFO  cn.analysys.ark.streaming.service.f -释放锁完成:/ark/streaming/lock/metadata/db_7eeefe9dbc34306e/event_write
2020-02-12 11:30:48.270 [RunnerStream-7eeefe9dbc34306e_event0] INFO  cn.analysys.ark.streaming.scheduler.main.f -Release lock success
2020-02-12 11:30:48.270 [RunnerStream-7eeefe9dbc34306e_event0] INFO  cn.analysys.ark.streaming.common.zk.a -删除zk节点[/ark/streaming/running/7eeefe9dbc34306e/ASK_EVENT_VVV_7eeefe9dbc34306e/0]成功

tail -300f /opt/soft/log/ark-streaming/${appkey}/profile0stream.log

2020-02-12 10:47:29.308 [RunnerStream-7eeefe9dbc34306e_profile0] ERROR cn.analysys.ark.streaming.bufferdb.kudu.utils.b -func singleSave error : org.apache.kudu.client.NonRecoverableException: MANUAL_FLUSH is enabled but the buffer is too big
	at org.apache.kudu.client.KuduException.transformException(KuduException.java:110)
	at org.apache.kudu.client.KuduSession.apply(KuduSession.java:93)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1035)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1043)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1043)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1043)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1043)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1043)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1043)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1043)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1043)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1043)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1043)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1043)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:901)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:836)
	at cn.analysys.ark.streaming.bufferdb.kudu.dao.a.addMapkey(ArkBufferDbKuduDataDao.java:138)
	at cn.analysys.ark.streaming.service.c.b(ArkServiceProfileImpl.java:760)
	at cn.analysys.ark.streaming.service.c.a(ArkServiceProfileImpl.java:376)
	at cn.analysys.ark.streaming.service.c.a(ArkServiceProfileImpl.java:328)
	at cn.analysys.ark.streaming.service.c.a(ArkServiceProfileImpl.java:215)
	at cn.analysys.ark.streaming.service.c.b(ArkServiceProfileImpl.java:161)
	at cn.analysys.ark.streaming.scheduler.main.f.a(StreamingRunningRunner.java:123)
	at cn.analysys.ark.streaming.scheduler.main.f.run(StreamingRunningRunner.java:69)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
	Suppressed: org.apache.kudu.client.KuduException$OriginalException: Original asynchronous stack trace
		at org.apache.kudu.client.AsyncKuduSession.apply(AsyncKuduSession.java:597)
		at org.apache.kudu.client.KuduSession.apply(KuduSession.java:79)
		... 25 more

2020-02-12 10:53:38.328 [RunnerStream-7eeefe9dbc34306e_profile0] ERROR cn.analysys.ark.streaming.bufferdb.kudu.utils.b -func singleSave error : org.apache.kudu.client.NonRecoverableException: MANUAL_FLUSH is enabled but the buffer is too big
	at org.apache.kudu.client.KuduException.transformException(KuduException.java:110)
	at org.apache.kudu.client.KuduSession.apply(KuduSession.java:93)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1035)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1043)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1043)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1043)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1043)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1043)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1043)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1043)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1043)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1043)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1043)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:1043)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:901)
	at cn.analysys.ark.streaming.bufferdb.kudu.utils.b.a(KuduUtil.java:837)
	at cn.analysys.ark.streaming.bufferdb.kudu.dao.a.addMapkey(ArkBufferDbKuduDataDao.java:138)
	at cn.analysys.ark.streaming.service.c.b(ArkServiceProfileImpl.java:760)
	at cn.analysys.ark.streaming.service.c.a(ArkServiceProfileImpl.java:376)
	at cn.analysys.ark.streaming.service.c.a(ArkServiceProfileImpl.java:328)
	at cn.analysys.ark.streaming.service.c.a(ArkServiceProfileImpl.java:215)
	at cn.analysys.ark.streaming.service.c.b(ArkServiceProfileImpl.java:161)
	at cn.analysys.ark.streaming.scheduler.main.f.a(StreamingRunningRunner.java:123)
	at cn.analysys.ark.streaming.scheduler.main.f.run(StreamingRunningRunner.java:69)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
	Suppressed: org.apache.kudu.client.KuduException$OriginalException: Original asynchronous stack trace
		at org.apache.kudu.client.AsyncKuduSession.apply(AsyncKuduSession.java:597)
		at org.apache.kudu.client.KuduSession.apply(KuduSession.java:79)
		... 25 more

2020-02-12 10:54:06.186 [RunnerStream-7eeefe9dbc34306e_profile0] ERROR cn.analysys.ark.streaming.bufferdb.kudu.utils.b -func addMapkey err ,xhwo is :妞妞
2020-02-12 10:54:09.834 [RunnerStream-7eeefe9dbc34306e_profile0] ERROR cn.analysys.ark.streaming.bufferdb.kudu.dao.d -errorsCount2::1
2020-02-12 10:54:11.834 [RunnerStream-7eeefe9dbc34306e_profile0] INFO  cn.analysys.ark.streaming.common.zk.a -准备开始创建节点/ark/streaming/stop/7eeefe9dbc34306e
2020-02-12 10:54:11.836 [RunnerStream-7eeefe9dbc34306e_profile0] INFO  cn.analysys.ark.streaming.common.zk.a -完成添加节点:/ark/streaming/stop/7eeefe9dbc34306e
2020-02-12 10:54:11.836 [RunnerStream-7eeefe9dbc34306e_profile0] INFO  cn.analysys.ark.streaming.common.zk.a -删除zk节点[/ark/streaming/schedule/ark1.analysys.xyz/waitrun/7eeefe9dbc34306e_profile_0_20200212104139]成功
2020-02-12 10:54:11.838 [RunnerStream-7eeefe9dbc34306e_profile0] INFO  cn.analysys.ark.streaming.common.zk.a -删除节点:/ark/streaming/schedule/ark1.analysys.xyz/running/7eeefe9dbc34306e_profile_0_20200212104139
2020-02-12 10:54:11.838 [RunnerStream-7eeefe9dbc34306e_profile0] INFO  cn.analysys.ark.streaming.common.zk.a -删除zk节点[/ark/streaming/schedule/ark1.analysys.xyz/running/7eeefe9dbc34306e_profile_0_20200212104139]成功
2020-02-12 10:54:11.838 [RunnerStream-7eeefe9dbc34306e_profile0] ERROR cn.analysys.ark.streaming.scheduler.main.f -java.lang.RuntimeException: flush data to kudu failed !
	at cn.analysys.ark.streaming.bufferdb.kudu.dao.d.flushData(ArkRowSetKuduImpl.java:137)
	at cn.analysys.ark.streaming.service.c.a(ArkServiceProfileImpl.java:1046)
	at cn.analysys.ark.streaming.service.c.a(ArkServiceProfileImpl.java:375)
	at cn.analysys.ark.streaming.service.c.a(ArkServiceProfileImpl.java:328)
	at cn.analysys.ark.streaming.service.c.a(ArkServiceProfileImpl.java:215)
	at cn.analysys.ark.streaming.service.c.b(ArkServiceProfileImpl.java:161)
	at cn.analysys.ark.streaming.scheduler.main.f.a(StreamingRunningRunner.java:123)
	at cn.analysys.ark.streaming.scheduler.main.f.run(StreamingRunningRunner.java:69)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

2020-02-12 10:54:11.838 [RunnerStream-7eeefe9dbc34306e_profile0] INFO  cn.analysys.ark.streaming.service.f -释放锁:/ark/streaming/lock/metadata/db_7eeefe9dbc34306e/profile
2020-02-12 10:54:11.838 [RunnerStream-7eeefe9dbc34306e_profile0] INFO  cn.analysys.ark.streaming.service.f -释放锁完成:/ark/streaming/lock/metadata/db_7eeefe9dbc34306e/profile
2020-02-12 10:54:11.838 [RunnerStream-7eeefe9dbc34306e_profile0] INFO  cn.analysys.ark.streaming.scheduler.main.f -Release lock success
2020-02-12 10:54:11.840 [RunnerStream-7eeefe9dbc34306e_profile0] INFO  cn.analysys.ark.streaming.common.zk.a -删除节点:/ark/streaming/running/7eeefe9dbc34306e/ASK_PROFILE_VVV_7eeefe9dbc34306e/0
2020-02-12 10:54:11.840 [RunnerStream-7eeefe9dbc34306e_profile0] INFO  cn.analysys.ark.streaming.common.zk.a -删除zk节点[/ark/streaming/running/7eeefe9dbc34306e/ASK_PROFILE_VVV_7eeefe9dbc34306e/0]成功

有用0评论0

高俊5个月前

你是不是属性特别多，而且都是字符串类型

有用0评论1

2

回答

项目的数据流一直开启中，转圈圈？

dohkoxiaozu 5个月前

数据流,Hive 168人浏览 1人关注复制链接

dohkoxiaozu4个月前

显示这个

有用0评论0

用户15549521443个月前

cd /data1

find kafka-logs/ -size 10485760c -name *.index -exec rm -rf {} \;

重启kafka问题解决！

有用0评论1

2

回答

数据流开启不了，怎么办？

朱楠@Argo运营 7个月前

数据流 166人浏览 0人关注复制链接

和数千用户一起探讨方舟的部署与使用提问与回答都将获得积分

官方文档帮助

积分榜更多>>

使用微信扫码登录方舟社区，随即提问

绑定手机

帐号密码不匹配

立即绑定

支持所有数据上报平台

你也可以在这些地方找到我们

易观微信公众号

免费

免费的智能用户运营工具

Argo是易观方舟的免费版，支持私有化部署。让你在使用方舟商业级产品能力的同时不考虑成本，免费搭建你的智能用户运营平台。

查看详情

Copyright © 2002-2020 易观版权所有京ICP备15065619号-1

服务热线: 4006 - 010 - 230 / 4006 - 010 - 231