Fast Search Internal Server error exception - Resolved - SharePoint 2010

Today’s article is based on the numbers of problems that I faced while configuring fast search. This is the first time I have done this configuration but there were string of problems I encountered while resolving the issue. I would like to share the complete troubleshooting steps so that your time will be saved, issue will be resolved, you can meet your deadline in case of any, reduce the frustration J

SP attributes:

- SharePoint Server 2010 with recent CU

- Fast Search Server 2010

Problem description:

I have configured the Fast Search and added the content sources. I was able to crawl but found out Search results are not coming / working.

Error Message:

"Internal Server error exception" while searching from SharePoint sites.

Troubleshooting done:

1.   Verified certificate installation and configuration on both FAST and SharePoint servers

Ref: http://technet.microsoft.com/en-us/library/ff381244%28v=office.14%29.aspx

2.   Examined Service application proxy, TCP Port status and IPV6 status of FAST and SharePoint servers

Ref: http://blogs.technet.com/b/rubeno/archive/2012/02/23/troubleshooting-the-connection-between-the-fast-search-server-contentdistributor-and-sharepoint-2010.aspx

3.   Verified that FAST Service Account is part of FASTSearchAdministrators and FASTSearchKeywordAdministrators group

Ref: http://technet.microsoft.com/en-us/library/ff599535%28v=office.14%29.aspx#BKMK_Useraccounts

4.   While troubleshooting FAST service and SharePoint Search Service status identified that SAMWorker process at FAST ADMIN Node was in DEAD state.

About SAMWorker:

Generates a security filter for each user to trim down the query results returned to a user.

Inputs  End user claims tokens from the QRProxyService. 

User security filter requests from the QRServer. 

Accepts configuration changes from the SAM Admin.        

Outputs Generates a user’s security filter which is joined with the user’s query in the QRServer to do security trimming.

Dependencies: LogService, SAM Admin Service, QRServer, QRServerProxy.

5.   We Provide Full Control for Machine Keys in the following location on FAST Server.

(location" “Documents and Settings\All Users\Application  Data\Microsoft\Crypto\RSA\ Machine Keys”)

Ref: http://msdn.microsoft.com/en-us/library/windows/desktop/bb204778%28v=vs.85%29.aspx

6.   After this we used a new browser and browsed to the site.

Got the error "The search request was unable to connect to the Search Service"

7.   When examined SharePoint ULS, was seeing error below,

02/24/2014 09:30:05.06 w3wp.exe (0x1CD8) 0x1364 SharePoint Server Search Query g4hr Monitorable FastSearchGateway::GetQueryResult--Communication Exception searching on QRServer: http://SharePoint.amolgalaxy:13287. Putting qrserver on the exclude list for a while. Exception: System.ServiceModel.FaultException: The server was unable to process the request due to an internal error.  For more information about the error, either turn on IncludeExceptionDetailInFaults (either from ServiceBehaviorAttribute or from the <serviceDebug> configuration behavior) on the server in order to send the exception information back to the client, or turn on tracing as per the Microsoft .NET Framework 3.0 SDK documentation and inspect the server trace logs.    Server stack trace:      at System.ServiceModel.Channels.ServiceChannel.ThrowIfFaultUnderstood(Message reply, MessageFault fault, String action, MessageVersion version, FaultConverter faultConverter)     at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc)     at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)     at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)     at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)    Exception rethrown at [0]:      at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)     at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)     at IProxyRemote.ProcessRequest(String request)     at Microsoft.SharePoint.Search.Extended.Query.Http.WcfSearchEngine.GetSearchResponse(IQuery query, String searchUrlPath, String urlQueryStr, Int32 timeout)     at Microsoft.SharePoint.Search.Extended.Query.Http.AbstractUrlBasedSearchEngine.SearchInternal(IQuery query, IResultParser parser, String urlStr)     at Microsoft.SharePoint.Search.Extended.Query.Http.AbstractUrlBasedSearchEngine.Search(IQuery query)     at Microsoft.Office.Server.Search.Query.Gateway.FastSearchGateway.GetQueryResult(IQuery query). f72be699-6dae-42d0-8bd6-501f61958e3d

02/24/2014 09:30:05.06 w3wp.exe (0x1CD8) 0x1364 SharePoint Server Search Query g4hk Monitorable AbstractSearchGateway::Search--Exception when searching for sharepoint site:"http://amolgalaxy/sites/admin-apps". Ex: Microsoft.Office.Server.Search.Query.SearchServiceNotFoundException: Service fault communicating with query service http://SharePoint.amolgalaxy:13287; The server was unable to process the request due to an internal error.  For more information about the error, either turn on IncludeExceptionDetailInFaults (either from ServiceBehaviorAttribute or from the <serviceDebug> configuration behavior) on the server in order to send the exception information back to the client, or turn on tracing as per the Microsoft .NET Framework 3.0 SDK documentation and inspect the server trace logs.     at Microsoft.Office.Server.Search.Query.Gateway.FastSearchGateway.GetQueryResult(IQuery query)     at Microsoft.Office.Server.Search.Query.Gateway.FastSearchGateway.ExecuteSearch(SearchRequest request, String queryString)     at Microsoft.Office.Server.Search.Query.Gateway.AbstractSearchGateway.Search(SearchRequest request) f72be699-6dae-42d0-8bd6-501f61958e3d

8.   We confirmed that the Proxy for Search is up and running by executing Get-SPServiceApplicationProxy

- Checked the proxy setting on the IE and was not configured.

- Tried to start the SamWorker (which is dead) but it failed to start

- In the event viewer found the following error

Log Name:      Application

Source:        FASTSearchSAMWorker

Date:         

Event ID:      0

Task Category: None

Level:         Error

Keywords:      Classic

User:          N/A

Computer:     

Description:

Service cannot be started. System.InvalidOperationException: Error Starting up.  Unable to initialize Startup Object Microsoft.SharePoint.Search.Extended.Security.Base.WCF.WCFServiceManager,Microsoft.SharePoint.Search.Extended.Security.Base"

at Microsoft.SharePoint.Search.Extended.Security.WorkerService.WorkerService.OnStart(String[] args)

at System.ServiceProcess.ServiceBase.ServiceQueuedMainCallback(Object state)

9. Followed steps below in an attempt to resolve this error,

- Stopped the Fast search service.

- Modified the Fast folder (%fastsearch%) permissions to add 'FASTSearchAdministrators' group members one by one, with full control.

- Set child objects to inherit permissions

- Removed the tilda files begining with ‘~’ from %fastsearch% folder

- Started the Fast services for SharePoint

It did not help resolve the issue. SamWorker remained in stopped state.

NCTRL Status   showing SAM worker  'dead'  state.

        Services.msc> FAST Search SAM Worker Service is not started. Manually startign it is not helping to  get sam worker running.

When ran commands below in given order,

 

-nctrl stop samworker

-nctrl stop samadmin

-nctrl start samadmin

-nctrl start samworker

   

The SAMWorker process gets in to running state and goes back to dead state in 2-3 minutes.

You are not able to get any search results in the SharePoint farm. Crawling and indexing works as expected.

This is a new deployment and the issue explained above was identified during the deployment phase.

-Checked %FASTSEARCH%/var/logs/syslog/ found only   'authorization-admin_amolgalaxy.txt' . So, this system never has SAMworker running.  If so, we should see at least authorization-worker_ amolgalaxy.txt in same folder.

-Tried to set FAST logging from warning to debug

 

Ran commands below at customer fast server,

 

-Set-FASTSearchSecurityLogLevel -DefaultLogLevel debug    --> Got error "cannot set log level."

 -Set-FASTSearchSecurityLogLevel -WarningNameSpaceLogLevel  debug --> Got error "cannot set log level.”

Checked the config file in location Config File(s) Location   and verified that they are good by comparing with a good FAST server's configuration files.

  For SAM ADMIN:

  %FASTSEARCH%/bin/Microsoft.Sharepoint.Search.Extended.Security.AdminService.exe.config 

%FASTSEARCH%/components/sam/admin  and

  For SAM WORKER

  %FASTSEARCH%/bin/Microsoft.Sharepoint.Search.Extended.Security.workerService.exe.config

%FASTSEARCH%/components/sam/worker

-Followed  http://support.microsoft.com/kb/2656160     and ran   "psconfig.ps1 -action p"   command via Powershell  as FAST For SharePoint  service account.  The command completed successfully.

-Registered ServicePrincipalNames for CN=s-fastdevsvc,OU=App Services,OU=CUSFA,DC=carrier,DC=UTC,DC=com  using SETSPN command.

That's it - Issue Resolved!  Verified that the search results are returned as expected.

If you have any queries/questions regarding the above mentioned information then please let me know, Thank you.

Operations on List and Libraries using SharePoint Designer 2010

SharePoint Designer is a very powerful tool and that’s why it being used for the customizations purposes! Apart from customizations, we can build variety of workflows by using SPD2010, which I will cover in a different article. Every GUI operation is possible using SharePoint Designer 2010. Frankly speaking it’s not possible to cover every aspect here but will share the major aspects which we required on regular basis, but if you are stuck at some point which is not a part of this article then please don’t hesitate and drop me a note by mentioning your query and we will discuss further!

1.   How to create a new list/library using SharePoint Designer 2010

2.   How to rename a list/library using SharePoint Designer 2010

3.   How to delete a list/library using SharePoint Designer 2010

4.   How to open list settings page by using SharePoint Designer 2010

5.   How to save the list as template by using SharePoint Designer 2010

How to create a new list/library using SharePoint Designer 2010

a)   Open the SharePoint Designer 2010

b)   Open your SharePoint Site ->it will ask you credentials

c)   Once the site structures opens in front of you then click on list and libraries from the left navigation see the following screenshot-

d)   Once the list and libraries gets highlighted then you can see the existing structure in the right side of the screen-> click on any list so that the top navigation options gets enabled/highlighted – please refer the following screenshot.

e)   Example you wanna create ‘custom list’, as you can see from the screenshot there are several list options are available.

f)    Click on any option directly and it will give you the option to type the name of the list and description

g)   Hit OK

h)   A list will be created successfully without any issues!

How to rename a list/library using SharePoint Designer 2010

a)   If you can refer the above steps then steps from a-d are same.

b)   Click on any list that you wanted to rename it by using SharePoint Designer 2010.

c)   Select Rename from the top.

d)   Type the name as per your choice/requirements.

e)   Hit OK. – Please refer the following screenshot.

How to delete a list/library using SharePoint Designer 2010

a)   If you can refer the above steps then steps from a-d are same.

b)   Click on any list that you wanted to DELETE by using SharePoint Designer 2010.

c)   Here we have multiple options by which you can easily implement this

d)   Right click on that and hit Delete OR

e)   Select the list and hit Delete from the top.

How to open list settings page by using SharePoint Designer 2010

a)   If you can refer the above steps then steps from a-d are same.

b)   Click on that list for which you wanted to open the list settings page by using SharePoint Designer 2010. – please refer following screenshot

c)   Once the list settings page get open then you can do anything like-

A.   Change the column metadata

B.   Delete the column

C.   Rename the column

D.   Create a new view

E.   Modify the view settings

F.    Apply filtering / sorting rules etc.

How to save the list as template by using SharePoint Designer 2010

a)   If you can refer the above steps then steps from a-d are same.

b)   Click on the list that you want to save as template

c)   Select the option ‘save as template’ from the top

d)   Please refer the following screenshot

e)   Enter the name

f)    Hit OK, that’s it.

Setup the home page using SharePoint Designer 2010

Today’s article is based on simple facts which can be done by GUI also but would like to share the way by using SharePoint Designer 2010.

Here are the complete steps:-

1.   Open the SharePoint Designer 2010 (Start->All programs->SharePoint->Microsoft SharePoint Designer 2010)

2.   Select open site option

3.   Enter the site URL without default.aspx / Home.aspx

4.   Enter the credentials i.e. your username and password

5.   Select the ‘site pages’ library/ the location where you have created the pages

6.   Right click on the page

7.   Select ‘Set as Home page’

Note- Not only Home.aspx but you can set any page as your welcome page by this process.

Please refer the following screenshot for better clarity!

Please let me know in case of any issues/questions regarding the above mentioned information. I would be more than happy to help you as well as resolves your issues, Thank you.

Technorati

4TDZUD7WJ6NH

You cannot delete a hidden column

Working on lot of tickets nowadays, trying to share as much as i can!

Today’s article is based on a simple issue but needs a quick trick to resolve it. I have resolved this issue within a quick span but it might be difficult for somebody to troubleshoot and will take lot of time to get stuck on the same. Just wanted to save your time by sharing this article, but still after following the below mentioned information, if your issue still remains unresolved then please drop a note in the form of comment and I would be there to help you!

Problem description

One of the users is trying to delete a column from a custom list but he is facing a very weird problem which might scare you initially, as hey what happening in my farm J

Error message

You cannot delete a hidden column. <nativehr>0x81020092</nativehr> <nativestack> </nativestack>

What’s the ultimate goal? – To delete the column!

Please follow these steps-

Resolution

a)   Open the list where you are trying to delete the column

b)   List settings

c)   Go to the columns section

d)   Click on the column that you want to delete

e)   Change the metadata of that column - example from single line of text to multiple line of text

f)    Click OK

g)   Click on that column again

h)   Hit Delete

i)     You would be able to delete that column without any problems.

If you have any queries/questions regarding the above mentioned information then please let me know. Thank you.