Network Management

Hello guys

I recently installed IMC at a customer and after the instalation was complete i received following critical error:

Failed to conenct to database imc_icc@127.0.0.1/icc_db. Reason: Connecting to the database failed.

Any ideas how to solve this? I did a basic installation.

Thanks in advance!

I'm having the same issue and haven't been able to figure it out. I have opened an HPE case but still hasn't been resolved.

I'm getting "Failed to connect to database" on several databases, at random times. Not sure if you are experiencing the same.

Hi Guys,

It happened to me time to time, but each time it was because my SQLServer service did not restart after a VM reboot.

But I assume you have already checked this.

Greetings

Ray

My issue seems to be occuring throughout the day, not even after a reboot. iMC seems to run fine, I just get random cirtical alerts throughout the day with that error "Failed to connect to database ..."

I checked to see if it was a password issue by running these commands on a couple of databases giving the alert but it is responding with a password .

pwdmgr.bat -query 127.0.0.1 aclm_db imc_aclm
pwdmgr.bat -query 127.0.0.1 invent_db imc_inventory

---

@jjacobs wrote:

I checked to see if it was a password issue by running these commands on a couple of databases giving the alert but it is responding with a password .

pwdmgr.bat -query 127.0.0.1 aclm_db imc_aclm
pwdmgr.bat -query 127.0.0.1 invent_db imc_inventory

---

Those commands will only be querying the local configuration file, they won't be querying the database. You need to use something that connects to the DB & runs a query. Do you have any sort of server monitoring system in use? Might be worth configuring that to monitor SQL. 

I would also check the SQL Server logs.

Ok, i was wondering if those commands actually queried the databases.  I checked our SQL logs and I'm not seeing any errors. Were running SQL Server ont the same server as iMC.

Quick Update. Spoke to a HPE tech today and it appears there are multiple people with this very issue.

We found the error by going to dmalog.txt

Under your IMC installation directory ($\Program Files\iMC\deploy\log\) and viewing the latest dmalog.txt. Matching up the times we were seeing the error he was able to identify the following error:

2016-09-27 11:58:45 [ERROR] [iMC-Database-Connect-Check] [com.h3c.imc.deploy.dma.monitor.DatabaseConnectMonitor::run(74)] Access database error
com.microsoft.sqlserver.jdbc.SQLServerException: The driver could not establish a secure connection to SQL Server by using Secure Sockets Layer (SSL) encryption. Error: "SQL Server returned an incomplete response. The connection has been closed. ClientConnectionId:7c006254-891a-4088-972a-645e9c2d37b6".
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.terminate(SQLServerConnection.java:1667)
    at com.microsoft.sqlserver.jdbc.TDSChannel.enableSSL(IOBuffer.java:1668)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:1323)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:991)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:827)
    at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1012)
    at java.sql.DriverManager.getConnection(DriverManager.java:582)
    at java.sql.DriverManager.getConnection(DriverManager.java:154)
    at com.h3c.imc.deploy.dma.monitor.DatabaseConnectMonitor.run(DatabaseConnectMonitor.java:74)
Caused by: java.io.IOException: SQL Server returned an incomplete response. The connection has been closed. ClientConnectionId:7c006254-891a-4088-972a-645e9c2d37b6
    at com.microsoft.sqlserver.jdbc.TDSChannel$SSLHandshakeInputStream.ensureSSLPayload(IOBuffer.java:651)
    at com.microsoft.sqlserver.jdbc.TDSChannel$SSLHandshakeInputStream.readInternal(IOBuffer.java:708)
    at com.microsoft.sqlserver.jdbc.TDSChannel$SSLHandshakeInputStream.read(IOBuffer.java:700)
    at com.microsoft.sqlserver.jdbc.TDSChannel$ProxyInputStream.readInternal(IOBuffer.java:895)
    at com.microsoft.sqlserver.jdbc.TDSChannel$ProxyInputStream.read(IOBuffer.java:883)
    at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:422)
    at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:460)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:863)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1188)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1215)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1199)
    at com.microsoft.sqlserver.jdbc.TDSChannel.enableSSL(IOBuffer.java:1618)

    He confirmed that is this same error that was found on other customers experiencing the same critical alerts. He did have a customer disable their windows firewall and that seemed to make the alerts stop. In my case windows firewall has already been disabled. He said they are going to do more research and should hopefully have a fix soon.

Well that was quick. Here is a workaround they gave me. Fingers crossed!

Here is the new workaround provided by our IMC Engineering, once this workaround is done - the previously removed/uninstalled Microsoft security KB (KB numbers below) can be re-installed.

Microsoft company provided another solution for those customers who have installed the latest updates. It can fix the issue without removing the installed updates. We have verified it in LAB, and it works fine.

To configure the SSL Cipher Suite Order Group Policy setting, follow these steps:

1. At a command prompt, enter gpedit.msc, and then press Enter. The Local Group Policy Editor is displayed.
2. Go to Computer Configuration > Administrative Templates > Network > SSL Configuration Settings.
3. Under SSL Configuration Settings, select SSL Cipher Suite Order, right click, then select Edit
4. Click Enabled, then go to the Options > SSL Cipher Suites, please move the cursor to find and delete the following two cipher suites (please see the attached picture ‘delete+new_cipler+suites.png’):

TLS_DHE_RSA_WITH_AES_256_CBC_SHA
TLS_DHE_RSA_WITH_AES_128_CBC_SHA

Note: It's not recommended to copy the string from 'Options >> SSL Cipher Suites' and edit it in other editor, e.g. notepad, because when you copy back, the length of the string will be limited.
5. Click OK to save
6. Reboot the physical server to take effect
 

IMC started to report this issue after MS Windows update of;

     For Windows 7 and Windows Server 2008 – MS update KB3172605 (replacement of KB3161608)

     For Windows 8.1 and Windows Server 2012 – MS update KB3172614 (replacement of KB3161606)

IMC Alarms are generated by the system process lsass.exe which is related with the Windows’ Security.  This is affecting IMC application (only), yet no IMC application user issues are reported -only intermittent IMC alarms are reported from different imc_** databases’.

Note: No changes/fixes in iMC for that issue currently.

Hi

Thanks for all this info, i really appreciate it.

I'll test this next week when i am back with the customer. Then i'll let you know what the results were.

Kind regards

Hi Jacob

Litlle question for you:

Can you share the picture they send you? (delete+new_cipler+suites.png). Just to be sure i do nothing rong. :)

Thanks again

Sorry doesn't look like my link worked.

https://s12.postimg.org/706cf34bh/delete_new_cipler_suites.png

I too have this same issue. This problem did NOT appear until I did the update to E0403P10 which is also a Critical Security Bulletin. 

Installing this has created more questions than answers.

I shutdown Database, hit the "install" button, go through all the hoops, and it seems to install.

Looking into the "Details" tab: A total of 16 items for deployment. but ONLY the first 7 actually install, stating "DEPLOYED". The rest of them state "For Upgrade". What?? Is this expected?? is this working as it should or completly foobarred.

Yet I can still connect and use the system and the interface seems to be the same?? So why are these other Updates...not installing?

And..since this "upgrade" I am too getting the DB connection error. 

Any insight welcome. 

Update:

I removed the TLS Cipher stuff and it seems like the DB connection error may have resolved. I will monitor for a day or two.

However, after that reboot, of course IMC wants to try and install the balance of stuff it couldn't before. Ok..I hit the button to continue.....the upgrade bombs as previously stated. 

"Batch deploy interrupted with error. Log file drive letter:\\program files\iMC\tmp\deploylog_1475090431881.zip"

Then it throws up a text file.....and with-in there...it looks like a bunch of duplicate key errors.

eg:

MSG 2627, Level 14, State 1, Server HP-IMC, Line 1

Violation of PRIMARY KEY constraint 'PK_tbl_perf_FEEC bla bla bla. CANNOT insert duplicate key in object 'imc.tbl_perf_template. The duplicate key value is (496)

Three errors all like this with different (values) then at the end:

The statement has been terminated.

(0 rows affected)    << this repeated 11 times

Any help would be appreciated. 

Hi jjacbos

I can tell you the problem is solved, by doing what you posted.

Thank you very much for the info and the help.

Kind regards

It seems that you NEED to shorten the string to bellow 1024 or else the setting won't survive after clicking ok/apply and reboot. At least not on a win2012r2 server.

Thx for the tip!

But seeing as it is a major pain to find these strings in the small GPE textbox, shouldn't it also work to copy the whole cipher-string to an editor, removing the two mentioned ones and then using a reg-file to modify the system?

I found the following 4 registry links which are changed by this GP setting:

• [HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Group Policy Objects\{F6ACAB91-2B97-4107-9017-0BA67B772CF0}Machine\SOFTWARE\Policies\Microsoft\Cryptography\Configuration\SSL\00010002]
• [HKEY_USERS\S-1-5-21-1978417431-1130399592-3233743443-1003\Software\Microsoft\Windows\CurrentVersion\Group Policy Objects\{F6ACAB91-2B97-4107-9017-0BA67B772CF0}Machine\SOFTWARE\Policies\Microsoft\Cryptography\Configuration\SSL\00010002]
• [HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Cryptography\Configuration\SSL\00010002]
• [HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Policies\Microsoft\Cryptography\Configuration\SSL\00010002]

But my guess would be, that only the last 2 entries are actually needed. So you add this line to them and it should work, right?

"Functions"="TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P384,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P384,TLS_DHE_RSA_WITH_AES_256_GCM_SHA384,TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA256,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P384,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P384,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P384,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P384,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P384,TLS_DHE_DSS_WITH_AES_256_CBC_SHA256,TLS_DHE_DSS_WITH_AES_128_CBC_SHA256,TLS_DHE_DSS_WITH_AES_256_CBC_SHA,TLS_DHE_DSS_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_3DES_EDE_CBC_SHA,TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA,TLS_RSA_WITH_RC4_128_SHA,TLS_RSA_WITH_RC4_128_MD5,TLS_RSA_WITH_NULL_SHA256,TLS_RSA_WITH_NULL_SHA,SSL_CK_RC4_128_WITH_MD5,SSL_CK_DES_192_EDE3_CBC_WITH_MD5"

Well, actually, even those 2 reg links are identical (they contain the same data from each other!), so one only needs to make this registry file:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Cryptography\Configuration\SSL\00010002]
"Functions"="TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P384,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P384,TLS_DHE_RSA_WITH_AES_256_GCM_SHA384,TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA256,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P384,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P384,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P384,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P384,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P384,TLS_DHE_DSS_WITH_AES_256_CBC_SHA256,TLS_DHE_DSS_WITH_AES_128_CBC_SHA256,TLS_DHE_DSS_WITH_AES_256_CBC_SHA,TLS_DHE_DSS_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_3DES_EDE_CBC_SHA,TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA,TLS_RSA_WITH_RC4_128_SHA,TLS_RSA_WITH_RC4_128_MD5,TLS_RSA_WITH_NULL_SHA256,TLS_RSA_WITH_NULL_SHA,SSL_CK_RC4_128_WITH_MD5,SSL_CK_DES_192_EDE3_CBC_WITH_MD5"

Hello

Yes it is the same with me, also several databases. :/