DGM-16979 is an common error and sometimes is misleading. One of the issue I faced in oracle 11g is, observer starts if you do manually even after login thru wallet.
dgmgrl /@wallet_sys
DGMGRL for Linux: Version 11.2.0.4.0 - 64bit Production
Copyright (c) 2000, 2009, Oracle. All rights reserved.
Welcome to DGMGRL, type "help" for information.
Connected.
DGMGRL>
When you try to start observer, it'll spit out errors like
Could not verify magic words.
DGM-16979: Unable to log on to the primary or standby database as SYSDBA
Even if you are using wallet successfully for other activities, you might still have to add default user and password to your existing wallet.
mkstore -wrl . -createEntry oracle.security.client.default_username sys
mkstore -wrl . -createEntry oracle.security.client.default_password PassWord
Now your observer should start.