POE port problems with my new Hikvision M-series NVR?

[John H]

Thanks @David
We and Hikvision Tech Support are trying to get to the bottom of the issue(s).
Once we/they have a clearer idea of the root cause we'll implement remedial action.
We will keep updating this thread.
Please allow us 48hrs - Monday madness, and the team have been hit by human bugs! (x2 WFH/off today)

Thank you for the update Phil. There's no urgency in my case. I'm just pleased the problem is actively being looked into for the benefit of several people on here. If that takes a while then so be it. And I know exactly what you mean. Life always has a habit of getting in the way when you're trying to do something.

David, you could well be right regarding the power supply. John A. has also suspected a power supply fault from very early on in this investigation. I don't have the technical equipment to log voltages so I'll leave that to the Hikvision engineers. I do have an old 7716NI 14/16P NVR. If that machine has the same power supply then I could try that in my new NVR to see if it resolves the problem. I know it's only a case of plugging a multi core plug into the main circuit board but I don't want to do anything which could compromise the warranty of my new machine.

John H.

 

[Kyle]

Thanks again everyone for your patience.

We are now seeing the same issues reported by @John A (below), @John H, and also in the thread that @David has linked:

The stability problem is when using the unit POE, the cams slow to a stop - then they speed up the catch up going by the clock but the more cams I connect the worse it gets, they then disconnect and come back.

The issues became evident almost immediately after installing 4 HDDs and fully loading the 16 POE ports with cameras.
Interestingly, we didn't see the same issues with just 1 HDD installed with all POE ports populated.

It looks like the cameras that are unstable are those on the left-hand side of the built-in switch (D1-D4 and D9-D12), just as @John H describes:

After several days of port swapping and testing the results, I can now confirm that any cameras connected to ports D1 to D4 and D9 to D12 are suffering random video loss of signal and sluggish operation in live view. And ports D5 to D8 and D11 to D16 work absolutely fine.

Also, only those cameras seem to be disconnecting (see the attached logs from last night and this morning).

I've updated Hikvision with videos showing this, so am just waiting for further instructions.

Sorry for the slow progress with this, it seems like this was a previously unknown issue - they should now be able to better understand the root cause to advise on the next action to take.

 

[Kyle]

We've been keeping an eye on the NVR we're testing and feel like the POE port issues may have been resolved with the latest firmware update (V4.61.410)...

As soon as we roll back to V4.61.400, the freezing becomes very evident again.

@John H - can you please try the newer firmware and let us know if you're seeing any improvement on your NVR?

 

[TheStork]

We've been keeping an eye on the NVR we're testing and feel like the POE port issues may have been resolved with the latest firmware update (V4.61.410)...

As soon as we roll back to V4.61.400, the freezing becomes very evident again.

@John H - can you please try the newer firmware and let us know if you're seeing any improvement on your NVR?

At my end I upgraded my DS-7716NI-M4/16P from V4.61.405 build 220930 that came as factory install, to the latest V4.61.410 build 221117, and I have this morning had a raft of video signal lost errors, but this time only on channel 1, rather than on a broader set of the ‘left side’ POE channels. So it seems the issue is still there, but perhaps slightly changed or maybe limited the number of ports impacted.

 

[Kyle]

Thanks for the update @TheStork - I haven't noticed any problems on channel 1 but it may be because it only has an old DS-2CD2343G0-IU connected, so I'll see if things are different with a more recent 8MP model.

Which camera are you using on channel 1, and are you able to check if the same problem exists with another camera on that channel, please?

 

[John H]

I updated my NVR to the latest firmware a week or so ago now and at first I thought there was an improvement but over the last few days I've had numerous camera disconnects and live view drop outs on port D12.

I'm still running 9 cameras. 8 of them on the working ports D5 to D8 and D13 to D16. The 9th camera is on port D12.

I haven't had the chance to swap around on the other ports to see if they're still showing the same errors but hopefully over the weekend I can experiment a bit and report back on my findings.

I'm still puzzled by the randomness of these error reports. On 24th Feb I had 35 errors on D12. The majority of those were 'IP Camera Disconnected' with a few 'Arming IP Channel Failed' mixed in between them. 28 errors the following day and 22 the day after that. And then 71 errors on the 28th Feb.

So, there's definitely no improvement on D12 since updating the firmware but as mentioned previously, I can't comment on the other ports because I haven't tried them yet. The Live View image of D12 is also constantly going off for a few seconds and them coming back on again.

John H.

 

[Kyle]

Thanks for checking @John H - I've just passed both updates above on to Hikvision, I'll be back with more info as soon as I can.

 

[TheStork]

Thanks for the update @TheStork - I haven't noticed any problems on channel 1 but it may be because it only has an old DS-2CD2343G0-IU connected, so I'll see if things are different with a more recent 8MP model.

Which camera are you using on channel 1, and are you able to check if the same problem exists with another camera on that channel, please?

Thanks Kyle. The camera I have on channel 1 is a DS-2CD2T42WD-I5. I haven’t been able to test with other cameras. With the previous firmware I saw the issue on channels 1-4, where the camera types used were DS-2CD2T42WD-I5 , DS-2CD2T32-I5 , DS-2CD2186G2-ISU.

 

[Kyle]

@John H - Are you happy for us to arrange for your NVR to be collected and returned to Hikvision for them to test themselves, please?

If you don't have another NVR that you can use temporarily, then we can ship out a temporary I-Series replacement to get you by.

They have been testing a couple of units and are struggling to replicate the issue, so need to investigate further with the one we know is problematic.

They originally asked if we could return the NVR we have that @John A sent back, but we were reluctant to do so, as this still seems better on the new firmware, so we would like them to test a unit that is showing obvious problems.

It's not yet clear if this is a batch issue, will be resolvable with new firmware, or is something that affects all of the 4-bay M-Series units - they've explained that this is what they're hoping to understand through testing.

They've assured me that they will test and return the NVR as a priority, and once they have a better idea of the root cause, be able to best advise on the next steps.

Sorry again for the slow progress with this everyone - until Hikvision can understand the problems fully, there's not much that we can do but wait.

 

[John H]

Hello Kyle,

Yes that is fine with me. I'm happy for my NVR to be returned to you for testing by Hikvision. I would appreciate a loan I series unit please as I've installed my previous machine at my parents house.

An update to my testing. Since updating to the latest firmware, I've tried a camera in ports D1 to D4 and D9. All are continuing to show the same problems so the firmware hasn't fixed anything for me.

I also thought that it might be a batch problem but a member of this forum based in the US was having similar problems. His machine would be a 120v model and therefore (presumably) from a completely different batch.

If you can advise me on the next steps to return my NVR, I'll make sure I get it disconnected, packaged up and ready to send.

Thank you.

John H.

 

[Kyle]

Great, thank you - I'll send you an email to get the collection of your NVR and loan unit arranged

Yours definitely sounds like one that it's worthwhile them testing - my guess is that there's some variable at play that causes the problems that we haven't worked out, rather than a batch issue as you mention.

 

[John H]

Thank you Kyle.

 

[dazulrich]

There is a new fw for the m series on the hik website (not on the UK portal yet).
v4.61.420_230217. no release notes as usual

 

[Kyle]

Thanks @dazulrich - I'll test it on one of our NVRs and try to get some more information or release notes on it from Hikvision.

I've been meaning to post an update on the other bugs reported in this thread but got distracted by the POE port issues, apologies...

1. Smart playback doesn't play a 5 second intro to the trigger event. It does if you draw a new line or area on screen but does not show the intro for any pre configured event. This is also happening on other models with recent firmware
2. I'm fairly sure (but could be wrong and haven't checked) that Smart Playback used to show markers for Alarm Input events. It doesn't anymore
3. Doing a Custom search and selecting 'Alarm Input' returns 'no results' despite alarm inputs being linked to 'trigger channel' and multiple events occurring (the events trigger correctly and 'notify surveillance centre' but do not appear listed in a custom search)

These have all been explained to Hikvision Support who have told me that these bugs have now all been reported back to their Chinese HQ to work on some fixes.
I pointed out that #3 is the priority to resolve, as it's effectively rendering the alarm input ports useless (hopefully #2 will be fixed at the same time).

4. Switching between Smart Playback and Normal Playback maintains timeline position for seamless playback. Switching between Normal Playback and Smart Playback does not - it jumps to the first event of the day instead of jumping forward to the next event after the previous Normal Playback timeline position. This was broken on all versions of I series after 4.40.017 and unfortunately has carried through to the M series. It is also broken on other NVR/DVR models and in some cases going between Smart and Normal playback tabs resets the playback position to midnight which is woeful if you're trying to continue watching something post event.​

So apparently, this change was deliberate, here's their response on this:

"When switching to Event playback, we hope that the clients won’t miss any events, so it will automatically jump to the start. If you require to change it, we can process you to the customization."​

I think it's unlikely that they'll change this back unless enough people ask for it (I did mention that I'd been asked about it a few times) - firmware "customization" is normally only available if you order a large quantity of an NVR and ask for specific changes to be made to the firmware.


There were also some separate issues in Hik-Connect that we've been looking at with a customer and Hikvision Support, which they were also going to fix in the next firmware release - it turns out that, certain firewalls will "filter the streaming signal" which was causing event playback in the app to become unstable - random event thumbnails would not have an image and unless stream encryption was disabled, they event footage would not play and instead give a "loading failed" error.

I'm yet to check, but I'm wondering if this is the main fix in V4.61.420...


Are you still seeing issues #1 and #2 above on this new version @JB1970?
I'll let them know that they haven't yet been resolved if so (I can see in your recent thread that #3 hasn't been fixed yet).

 

[JB1970]

Thanks @dazulrich - I'll test it on one of our NVRs and try to get some more information or release notes on it from Hikvision.

I've been meaning to post an update on the other bugs reported in this thread but got distracted by the POE port issues, apologies...


These have all been explained to Hikvision Support who have told me that these bugs have now all been reported back to their Chinese HQ to work on some fixes.
I pointed out that #3 is the priority to resolve, as it's effectively rendering the alarm input ports useless (hopefully #2 will be fixed at the same time).


So apparently, this change was deliberate, here's their response on this:

"When switching to Event playback, we hope that the clients won’t miss any events, so it will automatically jump to the start. If you require to change it, we can process you to the customization."​

I think it's unlikely that they'll change this back unless enough people ask for it (I did mention that I'd been asked about it a few times) - firmware "customization" is normally only available if you order a large quantity of an NVR and ask for specific changes to be made to the firmware.


There were also some separate issues in Hik-Connect that we've been looking at with a customer and Hikvision Support, which they were also going to fix in the next firmware release - it turns out that, certain firewalls will "filter the streaming signal" which was causing event playback in the app to become unstable - random event thumbnails would not have an image and unless stream encryption was disabled, they event footage would not play and instead give a "loading failed" error.

I'm yet to check, but I'm wondering if this is the main fix in V4.61.420...


Are you still seeing issues #1-3 above on this new version @JB1970?
I'll let them know that they haven't yet been resolved if so.

Hi Kyle

I never thought to check the other bugs so I've just had a look now:

1 - Smart Playback not playing 5 second pre event. This seems to be resolved for me. It's not only playing 5 secs before on events since updating the firmware, it's now adding the 5 seconds to events that occurred before the firmware update (I've gone back through some events from a few weeks ago and they all play correctly)

2 - I think this was my mistake as I've mentioned in another post this AM. Smart Playback only displays markers for motion, line, intrusion and not alarm input. (I re read the latest manual yesterday)

4 -

I think it's unlikely that they'll change this back unless enough people ask for it (I did mention that I'd been asked about it a few times) - firmware "customization" is normally only available if you order a large quantity of an NVR and ask for specific changes to be made to the firmware.

That's a bit disappointing, though they did tell me the same a long time ago when I spoke to someone Dan put me in touch with. If you're skipping through 25 or so smart events (skip unrelated video), you find the one you want and click to normal to view continuously. When finished, you would only ever want to go forward to the next event when you switch to smart playback again (as you've already reviewed the previous events). Anyway if you did want to go back over you only had to click the mouse on the timeline once. Anyway it is what it is...

 

[TheStork]

On the PoE / video signal loss problems, one other observation in case it is useful for hunting down the cause:
When the Video Signal Lost event occurs, and it sends a notification email, it seems to sometimes also lose the name of the camera.
Usually the email contains a line including the name of the camera with the error, in this example case “Front (east)” on channel 1:
CAMERA NAME(NUM): Front (east)(D1)

But at times it seems to lose / temporarily forget the name, instead with the error notification email for the same camera having a more generic / default name:
CAMERA NAME(NUM): IPCamera 01(D1)

It’s curious that it randomly temporarily forgets the camera name when it sends the video signal lost messages, given it should be the same type of event and same process of sending a notification. Not sure if it is related to the root cause of the loss of video signal, but thought I’d share the observation.

 

[Kyle]

Just a quick update: Hikvision has confirmed they are seeing the same POE / connectivity issues now that they're testing the NVR that @John H sent back to them.

They will give me a more thorough update once they have spent some more time with the NVR to understand the cause, so I'll be back with that as soon as they do...

 

[John H]

Thank you for that update Kyle.

I'm pleased they're seeing the same faults. We all now await their findings which hopefully lead to a successful remedy soon.

 

[johnfitzy]

It's not yet clear if this is a batch issue, will be resolvable with new firmware, or is something that affects all of the 4-bay M-Series units - they've explained that this is what they're hoping to understand through testing.

Are the 2-bay M-Series units also experiencing these POE port problems?

 

[JB1970]

Are the 2-bay M-Series units also experiencing these POE port problems?

I hope not. I've not seen any reports on here. My DS-7616NI-M2/16P with 9 channels populated has been fine (installed Nov 28th '22)