Monday, February 20, 2017

Pre Installation Setup for Hyperion EPM V11.1.2.4

1.  General

Recently, when I was working with Norwegian courier services leader for installation and configuration of Hyperion EPM V11.1.2.4, I got chance to document my steps and hence finally got chance to share on blog.
As they always say,
“The will to win is worthless if you do not have the will to prepare.”
To begin with Hyperion EPM V11.1.2.4 installation and configuration, we should prepare our environment and resources well for it.
That’s my aim here, to provide steps for pre-installation setup for Hyperion EPM V11.1.2.4. 

2.  Pre-Installation Checklist

1.       Work Area
·         Internet Access – Outside firewall (Disable firewall for the working folder – e.g. installers and oracle folder.
At beginning, we will have only installers folder, disable firewall for this folder.)
·         Computer within EPM Servers network
2.       Third party licenses
E.g. web server licenses, Java

3.       Software
Check all installers are downloaded from oracle edelivery

4.       Required documents
Check if all the required documents are downloaded from oracle edelivery 

5.       Preparing the Hardware
·         Confirm your plan of deployment architecture – whether single server or distributed environment for EPM
·         Plan which components to be installed on which servers if we are going for distributed environment.
·         Confirm whether computers meet hardware system requirements. Check following matrix.
·         Resolve Firewall problem
Open a restricted range of ports in your firewalls for client to server or server to server communication.
·         Disable antivirus
Exclude the EPM Oracle home directory from automatic antivirus scans and scan this directory only at scheduled times.

3.  Preparing Environment

Preparing ports

This section contains information about default port numbers for EPM System products.

Necessary ports should be enabled on EPM Environment servers. Firewall should be disabled at these ports for communication between servers or between servers and clients.




EPM Module
Default Port Number
Default SSL Port
Weblogic Administration Server
7001
Oracle Enterprise Manager Java Web Application Port
7001
SOA Server Port
8001
Foundation Services Web Application
28080
28043
Oracle HTTP Server
19000
IIS
80
443
Provider services (SmartView)
13080
13083
Essbase Administration Services
10080
10083
Essbase Agent
1423
Essbase server applications (ESSSVR)
32768–33768
Essbase SSL Agent
6423
OPMN (Oracle Process Manager and Notification Server) Local Port
6711

OPMN (Oracle Process Manager and Notification Server) Remote Port
6712

Essbase Studio Listen Port
5300

Essbase Studio remote Port
12080

Reporting and Analysis Framework Agent
6860

Reporting and Analysis Framework Agent RMI
6861

Reporting and Analysis Framework Services
6800–6805

Financial Reporting Web Application
8200
8243
Financial Reporting Default Service Ports
8205-8228

Reporting and Analysis Framework Web Application
45000
45043
Web Analysis Web Application
16000
16043
Financial Management Java Web Application Port
7363
7365
Financial Management Server port
9091
9092
FM Datasource Start Port
10001
10020
Planning Java Web Application Ports
8300
8343
Planning RMI Server Port
11333


Disable UAC on all servers

UAC must remain disabled in order for EPM System server components to function properly.
UAC can be enabled on end-user client desktops.
In Windows 2008 environments, disable User Access Control (UAC) on each server in the deployment.
Control Panel à System and Security

Server Preparations

Server Name Information

Web server(s) (could be more than one):
·         Name: _____________________________
·         IP: _________________________
Application server(s) (could be more than one):
·         Name: __________________________
·         IP: _____________________
Database server (information depending on Oracle DB or MS SQL):
·         Name: __________________________
·         IP: _____________________
·         SID/Instance/Port: _____________________

Operating System

Prepare each server for the installation:
Update server software as needed.
·         For each server in the deployment, apply Windows updates and reboot before starting with installation and configuration of Hyperion EPM System
·         Include service packs, hot fixes, fix packs, etc.
·         Always stay on the latest OS Service pack level
·         Install Internet Explorer (IE) on the servers,
For 11.1.2.4.x any version higher than IE 9
·         Disable unnecessary services.
·         Power option set to High Performance

Synchronize clocks

The clock on each server must be synchronized to within one second difference.

Resolving Host Names

The canonical host name of each server must be the same when accessed from within the server and from other servers in the deployment.
You can modify hosts file located at following location C:\Windows\System32\drivers\etc
 epmsys_hostname.bat
An archive of the utility (epmsys_hostname.zip) is available in the directory where you unzip the assembly for EPM System Installer.

Disabling Anti-virus Software

Antivirus software can cause performance issues with EPM System products if, each time you access any resource on the server, the antivirus software tries to open and scan the object. To prevent these issues, exclude the EPM Oracle home directory from automatic antivirus scans and scan this directory only at scheduled times.

Excluding the EPM Oracle home directory from automatic antivirus scans 
Scan EPM Oracle home directory only at scheduled times.   

Web Server

·         Make sure that a DNS (CName) name is ready for the load balancer (BIG IP)
·         Make sure that the load balancing solution is in place for the web server(s). (Server group 1)
·         The following products require IIS to be installed with ASP.NET and ASP support enabled before configuration of the EPM System product: (Server group 1)
Strategic Finance

Network Share

·         A network share for storing software for installation together with patches and used 3rd party components
·         A read/write network share needs to be created for RAF to store the Workspace objects (Server group 1)
o   Name on share: RAF
o   Network path/address to share: \\ServerName\D$\EPMQA\
·         A read/write network share needs to be created for LCM to store the application artifacts* (Server group 1)
o   Name on share: LCM
o   Network path/address to share: \\ServerNamel\D$\EPMQA\LCM
Start with 200 GB for all three products and make sure that the shared folder is locked down to install account to start with.

Preparing User Account

·         Do not use the Administrator user to install and configure. Run EPM System Installer and EPM System Configurator as a user with administrator rights.
·         When you upgrade, apply a maintenance release, or patch this server, use the same user account that was used to install and configure the earlier release.
·         Make sure that no password policy forces the user account to change password or expire.

Preparing External Directory Lookup Account

Prepare a service account for external authentication against the authentication provider of your choosing (MSAD or LDAP). This account allows the Shared Services component to make a browse level connection to your chosen authentication provider. The purpose of this is so that you can use accounts from your Authentication provider to log in to the Oracle Hyperion EPM
Products
·         Create a service account with browse level privileges for MSAD or other LDAP Provider
·         Ensure service account does not have any special characters
·         Ensure service accounts Distinguished Name (DN) is able to access MSAD or other LDAP Provider
·         Ensure that MSAD or other LDAP Provider port is known
·         Be familiar with the name of a Primary Domain Controller that has access to MSAD and the base DN
·         Have information ready to the consultant for where the users and groups is in the directory so that configuration to the correct OU can be done
·         Ensure that server can communicate with MSAD or other LDAP Provider

MSAD/LDAP information

·         Name of connection: ________________ (identifier we should user)
·         Server name: ____________________
·         Port number: _______________________
·         User ID: ___________________ (The user to log in to directory service)
·         Password: ____________________
·         Attribute for user ID : ____________________ (the username in EPM)
·         Base DN for lookup user: __________________________
·         Base DN for user folder: ___________________________
·         Base DN for group folder: __________________________
·         Filter for EPM users: _____________________

Check disk space and RAM

Client System Disk Space and RAM

The recommended RAM requirement for all clients is 1 GB.
The installation program checks for twice the required disk space, based on your product installation choices.

Server Disk Space and RAM



4.  Preparing Database

Oracle Database Client

Unlike previous versions of Hyperion EPM system till V11.1.2.2, Oracle Database client comes bundled along with Hyperion EPM v11.1.2.4 installers.
Under Foundation services, we have 32 bit Oracle DB client and 64 bit Oracle DB client.

We can either use the bundled client or we can install DB client separately. In that case we need to uncheck Oracle client under Foundation services.

If we are installing DB client separately, we need to refer following points.

Separate DB client

Install the full Oracle Database client on the following machines before you start your installation of EPM System products:
·         Performance Management Architect Dimension server
·         Financial Management application server
·         FDM Application Server and any machine that has FDM Workbench
·         Strategic Finance
For v11.1.2.4 installation, we need to install DB client. Following are some bullet points to be highlighted.
·         We need to install Oracle DB client (Both 32 bit and 64 bit) on HFM and FDM server if we are using distributed environment.
Reason – HFM is 32 bit application and for running all processes properly, we need both 32 as well as 64 bit DB client.
·         For Foundation server, we can install only 64 bit client. 32 bit client can be skipped.
·         While installing client, 32 bit DB client is first installed followed by 64 bit client installation.

Oracle Database

The database must be created using Unicode Transformation Format UTF-8 encoding (character set).

Make sure that the following attributes and settings are used for the Oracle database:
·         alter system set PROCESSES=1000 scope=spfile;
·         alter system set SESSIONS=2000 scope=spfile;
·         alter system set SESSION_CACHED_CURSORS=200 scope=spfile;
·         RESIZE 1000M;
·         AUTOEXTEND ON
·         NEXT 500M
·         MAXSIZE 5000M;
·         SEGMENT SPACE MANAGEMENT = AUTO
Product Specific
Oracle Hyperion Financial Management
·         alter system set OPEN_CURSORS=5000 scope=spfile;

Database Privileges

The following privileges must be granted to the owners of the database schemas:
·         CREATE SESSION
·         CREATE VIEW
·         CREATE TYPE
·         CREATE TABLE
·         CREATE CLUSTER
·         CREATE TRIGGER
·         CREATE SEQUENCE
·         CREATE INDEXTYPE
·         CREATE PROCEDURE
·         CREATE ANY SYNONYM
·         DROP ANY SYNONYM
·         UNLIMITED TABLESPACE








Purpose of schema
QA
Initial Size (Auto Extend)
DB Name (SID)
Q_EPMDB
Shared Services
Q_PLN4
1 GB
Reporting and Analysis Framework
Q_RAF
1 GB
HFM schema
Q_HFM
30 GB
Calc Manager
Q_CALC
500 MB
Planning configuration
Q_PLN
100 MB
Planning application 1
Q_RP01
100 MB
Planning application 2
Q_DInput
100 MB
Planning application 3
Q_PLN3
100 MB
Planning application 4
Q_PLN2
100 MB
Essbase Administration Services
Q_EAS
100 MB
Essbase Server Studio
Q_ESS
100 MB
FDMEE
Q_FDMEE
30 GB
Strategic Finance
Q_SF
500 MB
Extra schema
Q_EX1
500 MB
Extra schema
Q_EX2
500 MB

2.  Web Application and Web Servers

Web Application Server

WebLogic Server

Oracle provides a limited-use license of WebLogic Server for use with EPM System products.

Web Server

Oracle HTTP Server

Bundled with EPM installer and can be installed during installation of foundation services.

IIS Server

None of the components except HSF require IIS as web server. So we require to install IIS.






Wednesday, February 8, 2017

Monitoring Mechanism for Hyperion EPM V11.1.2.4

General

I was happy when we successfully implemented BACKUP mechanism for Hyperion EPM V11.1.2.4 system for the first client which was Logistic company from France. The further discussion popped up few more queries like "How many FTE's for monitoring and maintaining the Hyperion System?", "What kind of automation for monitoring tasks?"bla bla.
This expedited my thought process and finally we proposed and implemented adeptly a proper Monitoring mechanism for this client. And yes, the process followed for our successive clients from various domains - specifically - Oil and Gas company from Dubai, Courier Services Leader from Norway and Donuts chain from USA.
As the famous astronomer and author Carl Sagan has said:
Absence of evidence is not evidence of absence.

So might be the case that right now, we do not have an evidence that there is no issue with our Hyperion EPM V11.1.2.4 but there are some hidden issues present and we are ignoring due to lack of evidence.

So, here I am, trying to outline the few monitoring and maintenance activities to properly maintain and support Hyperion EPM V11.1.2.4 from an IT prospective.

EPM V11.1.2.4 Backup Steps

Hyperion Services Monitoring

All the services responsible for Hyperion EPM V11.1.2.4 application should be monitored regularly.
If any service goes down, that will generate an alert for Hyperion Admin group via email.

Hyperion Services Monitoring
Server
All Hyperion servers
WebSvr1, WebSvr2, AppSvr1, AppSvr2, EssSvr
Services to be monitored
Hyperion Services on all above servers
Action 
Critical alert call – tickets for normal - if service goes down

Services to be monitored:
WebSvr1 and WebSvr2
Name                                                                                                                          Startup
Oracle Hyperion CALC Manager Java Web Application                                            Manual
Oracle Hyperion Financial Management – Web Tier                                                   Manual
Oracle Hyperion Financial Reporting Java Web Application                                      Manual
Oracle Hyperion Foundation Services Managed Server                                              Manual
Oracle Hyperion Reporting and analysis Framework Java Web Application              Manual
Oracle Hyperion Reporting and analysis Framework                                                  Manual
Oracle Process Manager (ohsinstance 1649849633)                                                   Manual

AppSvr1 and AppSvr2
Name                                                                                                                          Startup
Oracle Hyperion FDM Enterprise edition Java Web Application                               Manual
Oracle Hyperion Financial Management – Java Server                                              Manual
Oracle Hyperion Planning Java Web Application                                                       Manual
Oracle Hyperion Provider Services Java Web Application                                         Manual
Oracle Hyperion RMI Registry                                                                                    Manual
Oracle Hyperion Strategic Finance Java Web Application                                          Manual
Oracle Hyperion Strategic Finance Server                                                                   Manual

EssSvr
Name                                                                                                                          Startup
Oracle Process Manager                                                                                             Manual
Oracle Hyperion Essbase Studio Server                                                                     Manual
Oracle Hyperion Administration Services Java Web Application                              Manual

Hyperion Logs Monitoring

Hyperion log analysis report

Log Analysis Report
Server

All Hyperion servers
WebSvr1, WebSvr2, AppSvr1, AppSvr2, EssSvr
Frequency
Daily
No of Backups before purging
30
Action
Send email to Hyperion admin with report as attachment only if there is error in report

Log Analysis utility is provided with V11.1.2.3 onwards.
But this is standalone utility provided in patch - - and can be used with previous versions as well.
PFA introductory demo of this utility.

Command = loganalysis.bat -system -d e:\Oracle\Middleware -o total_report
We should include this command to run once every day and send report to Hyperion if there is any error in report.

Periodic Cleanup of logs

Periodic cleanup of logs
Server

All Hyperion servers
WebSvr1, WebSvr2, AppSvr1, AppSvr2, EssSvr

Frequency
Weekly
Action
Send email to Hyperion admin with report of logs modified after deletion

It’s important to keep the system clean. The Hyperion logs can get huge if left alone. Large log files can seriously impact performance as well. Periodic log cleanup is needed
Check logging.xml setting for each module of hyperion on ALL servers and modify to delete files after the size goes beyond 1GB

Sample logging.xml
<?xml version="1.0" encoding="UTF-8"?>
-<logging_configuration>
-<log_handlers>
-<log_handler class="oracle.core.ojdl.logging.ODLHandlerFactory" name="hfminterop-handler">
<property name="path" value="${logging.folder}/InteropJava.log"/>
<property name="maxFileSize" value="50000000"/>
<property name="maxLogSize" value="1000000000"/>
<property name="deleteFiles" value="true"/>
<property name="useSourceClassAndMethod" value="false"/>
<property name="useRealThreadId" value="true"/>
</log_handler>
</log_handlers>
-<loggers>
-<logger name="" useParentHandlers="false" level="INCIDENT_ERROR:1">
<handler name="hfminterop-handler"/>
</logger>
</loggers>
</logging_configuration>

Periodic Cleanup of application artifacts

Periodic cleanup of logs
Server

All Hyperion servers
WebSvr1, WebSvr2, AppSvr1, AppSvr2, EssSvr
Frequency
Weekly
Action
Send email to Hyperion admin with report of logs modified after deletion

It’s important to keep the system clean. Over the years, we accumulate stale an unused reports, applications, data, etc. It’s especially important to ensure you remove and de-provision any users that are no longer valid. I recommend a quarterly cleaning. Stale security, applications, and data can impact performance and make upgrades messier.

Patches/Upgrades

Patches are released all the time. It important to keep updated on what’s out there. Just make sure you do not just simply install patches simply because they are out there. You should apply only those that exactly match issues and the products/configuration you have.
 
Patches/Upgrades
Server

All Hyperion servers
WebSvr1, WebSvr2, AppSvr1, AppSvr2, EssSvr
Frequency
Semester
No of Backups before purging
Complete
Action
Ticket for Admin for analyzing new Hyperion patches and applying in system


Nightly jobs

Nightly jobs
Server

All Hyperion servers
WebSvr1, WebSvr2, AppSvr1, AppSvr2, EssSvr
Frequency
Daily
Action
Send email to Hyperion admin with report as attachment
In almost every case there are nightly data loads, calculations, backups, consolidations, etc that run and these jobs need to run on time. We must be alerted if jobs fail. Monitoring, automating, and resolving issues with nightly jobs can be time consuming depending on the complexity and frequency.

Manual Health Monitoring of application

Manual Health Monitoring
Server

All Hyperion servers
WebSvr1, WebSvr2, AppSvr1, AppSvr2, EssSvr
Frequency
Daily
Action
Send email to Hyperion admin with report as attachment

An IT staff should be prepared to periodically check the health of the system by fully logging in, checking logs, services, process, memory utilization and other system resources, etc. Of course this can be done programmatically using standard industry monitoring tools as well.

Periodic IT-Finance Meetings

IT-Finance Meetings
Server
NA
Frequency
Monthly
Action
Send MOM to all participants with action plan
I recommend a monthly meeting between the business users of the system and IT. It’s important to talk about general issues, service levels, root cause analysis of historical problems, performance, and what’s on the horizon for changes coming up.
Seem like a lot? It is. Is it a full time job? Depends. Either way, day-to-day maintenance activities like these can be a killer on today’s IT administrators that are supporting many enterprise systems at a time, not to mention implementing new projects constantly.
One option to reduce cost and risk but to ensure these activities are always being done is to leverage our Managed Applications group.  It is an affordable and flexible way to meet whatever service needs you have.