Showing posts with label Dev. Show all posts
Showing posts with label Dev. Show all posts

Saturday, November 22, 2014

Using Lucene search in DotNetNuke / Evoq Content 7+

Intro

I've recently improved a DNN module to use the new search engine present in the latest version of DotNetNuke. Before, I implemented my own search tool using SQL queries, but it was not very efficient (with LIKEs) and users always found ways not to find what they where looking for (using diacritic and other special characters, using quotes a.s.o...). In short, it was not good ;-)

I googled a bit to find examples, but didn't find many. So instead, I looked up the in the DNN source code. Here is a summary of my findings...

There were 2 areas I wanted to cover:

  • submit items to the DNN search engine (there are examples of that);
  • use the engine to retrieve my items and display them in my module (didn't find any example);


Submitting items to crawl

To use Lucene, you have to use the latest "ModuleSearchBase" base class for your FeatureController instead of implementing the ISearchable interface as in older versions.
You then override the GetModifiedSearchDocuments method and return a list of SearchDocument:

using System;
using DotNetNuke.Entities.Modules;
using DotNetNuke.Services.Search;
using DotNetNuke.Services.Search.Entities;
using DotNetNuke.Services.Search.Internals;

public class FeatureController : ModuleSearchBase
{
    private static readonly int ModuleSearchTypeId = SearchHelper.Instance.GetSearchTypeByName("module").SearchTypeId;

    public override IList GetModifiedSearchDocuments(ModuleInfo oModuleInfo, DateTime beginDate)
    {
        var oDocuments = new List();
        var oDocument = new SearchDocument();
        oDocument.Keywords.Add("keyword1", "value1");
        oDocument.UniqueKey = "some unique key to your app";
        oDocument.Title = "Title of the document";
        oDocument.Body = "Content of the document...";
        oDocument.AuthorUserId = oModuleInfo.LastModifiedByUserID;
        oDocument.ModuleId = oModuleInfo.ModuleID;
        oDocument.ModuleDefId = oModuleInfo.ModuleDefID;
        oDocument.PortalId = oModuleInfo.PortalID;
        oDocument.TabId = oModuleInfo.ParentTab.TabID;
        oDocument.SearchTypeId = ModuleSearchTypeId;
        oDocument.QueryString = "Parameters to pass to the page where my module is inserted into";
        oDocument.ModifiedTimeUtc = DocumentLastModifiedDate;

        // Important, if false, the document will be deleted from the search index
        oDocument.IsActive = true; 

        oDocuments.Add(oDocument);
        return oDocuments;
    }
}

You then run the site crawler job (or wait for it to run automatically) and you should be able to find your documents in the standard search. Clicking on an item will call the page where the module is located and add the querystring you specified for each document.

Note: you should take the "beginDate" parameter into account to only return new and modified documents, otherwise duplicate entries will show up. Unfortunately, if you purge the index in DNN, that date is not updated, and your module will return modified entries instead of returning all of them to refill the index... This is a bug I have reported to support, hopefully it will be fixed someday.

Using the DNN search in a module

A query is very simple to make. You make a new instance of SearchQuery, fill in some data, run it and display the results:

using DotNetNuke.Services.Search.Controllers;
using DotNetNuke.Services.Search.Entities;

protected void btnSearch_Click(object sender, EventArgs e)
{
    var oSearchQuery = new SearchQuery();
    oSearchQuery.KeyWords = "Search query";
    oSearchQuery.ModuleId = this.ModuleId;
    oSearchQuery.TabId = this.TabId;
    oSearchQuery.PortalIds = new List() { this.PortalId };
    oSearchQuery.WildCardSearch = true;

    var results = SearchController.Instance.ModuleSearch(oSearchQuery);

    if (results != null)
    {
        if (results.Results == null || results.Results.Count == 0)
            plhSearchResults.Controls.Add(new LiteralControl("<div class='alert alert-info'>" + LocalizeString("NoSearchResults") + "</div>"));
        else
        {
            plhSearchResults.Controls.Add(new LiteralControl("<ul style='list-style-type: none;'>"));
            foreach (var oRes in results.Results)
                plhSearchResults.Controls.Add(new LiteralControl(string.Format("<li><a href='{0}'>{1}</a><p>{2}</p></li>", oRes.Url, oRes.Title, oRes.Snippet)));
            plhSearchResults.Controls.Add(new LiteralControl("</ul>"));
        }
    }
}

Where plhSearchResults is an asp:placeholder receiving the list of results.

Hope that helps!

Thursday, March 6, 2014

ASP.Net Web API + OData + $inlinecount

So, I had a nice single page application querying data from a Web API backend. As I wanted to allow searching and paging, I used the OData extensions and all was well. Until I needed to know the total number of items in the dataset after having it filtered using $filter, $top and $skip. At the time, I didn't find any easy way to do it, so I implemented a second query to the API to return the total number of items. Problem is, this second query returned the total number of items and didn't take into account any filtering I had applied in the OData query (using $filter). I let that dormant for a while...

Then came the need to have a nice navigation footer, and I used this component: http://botmonster.com/jquery-bootpag/

This component just needs to know the number of pages and at which page it currently is. But then again, we need to know the number of items to know the number of pages to display. Back at square one ;-)

I looked for ways to implement this once again and found more info this time (technology matured or better search query in Google?). Some articles suggested to force the verbose mode of OData (by using "&$format=verbosejson" or adding an accept header with a value of "application/json;odata=verbose"), to no avail. I would always receive an array with the X items I requested using the $top parameter and nothing else.

Until I came across these 2 posts on SO:
http://stackoverflow.com/questions/18428763/web-api-odata-inlinecount-not-mapped
and http://stackoverflow.com/questions/18197041/reconstructing-an-odataqueryoptions-object-and-getinlinecount-returning-null

In short, here is what I had to change:

[Queryable(AllowedQueryOptions = AllowedQueryOptions.All)]
public IQueryable Get()
{
return m_oItems.AsQueryable();
}

changed to:

public PageResult Get(ODataQueryOptions options)
{
IQueryable results = options.ApplyTo(m_oItems.AsQueryable());
return new PageResult(results as IEnumerable, Request.GetNextPageLink(), Request.GetInlineCount());
}

Then in my JS file, I added "&$inlinecount=allpages" to my query and where I would treat the data blob returned by $.getJSON as an array, I simply use data.Count to have the number of items that match my query, and data.Items as my array of items.

The solution looks so simple now, but took me some time to figure out. Hope this helps...

Friday, February 22, 2013

viashopia Android app

My first mobile application, made using PhoneGap 2.3.0, is available on the Google Play Store:



Also visit the brand new web site of viashopia here: http://www.viashopia.com

Thursday, May 28, 2009

Caching portlets in ALUI 6.x

I've been developping portlets (gadgets) for nearly ten years now, starting with Plumtree Portal Server 3.5. A the time, there was a very nice document called "The Gadget Book" with all the details about the task of developping gadgets as they were called then. This PDF has been replaced by other versions since, but I still remember a few good practices for having a fast and reliable portal.

Among other things, there was the caching strategies to implement on the portlet side. The portal uses the standard HTTP mechanisms for calling content from the portlet server, as described in RFC 2616. Using the HTTP ETag and Last-Modified headers, we could prevent rendering a whole portlet when its content would remain unchanged. It proceeded like this:

- First call of the portlet by the portal. No special header is passed;
- The portlet returns the content to display, and sets the ETag and/or Last-Modified header;
- On the next access to the portlet, when the minimum cache time specified in the portlet configuration is over, and the maximum time not being reached, the portal calls the portlet giving back the content of the previous header;
- The portlet checks if the content has changed since the last call (in my case by comparing the timestamp passed in the Last-Modified header with a timestamp stored in a DB). If the content should be regenerated, I send the full content and give the new value for the headers. If not, I simply return an HTTP error 304 (Not Modified), and the portal would in that case display the content stored in its cache.

This worked perfectly for ages, until version 6.0 was released. On that version, when the 304 error was returned, the portal would display an error instead of displaying the cache (even when the setting "Suppress errors where possible (show cached content instead)" was checked.

I had several emails going back and forth with the Plumtree/BEA support and they finally acknowledge this as a bug. But in version 6.5, which is the one we are currently running (on http://www.myschool.lu/), that bug is still unresolved.

So, I disabled that part of my code, waiting for a solution to come eventually. I'm still waiting ;-) And my portlet is used in even more places than before (it displays content stored in a DB in many community pages) and the cache is key to allowing proper rendering times. The content would change once in a while, but hundreds of users would see the unchanged content in the meantime.

So, I tried to find a solution to this, and I have implemented the following workaround:

- Configure the portlet in this way (like in old times when caching worked):

- In my caching routine, I check the "CSP-Aggregation-Mode" header to see if I'm called as a portlet in a page, or as a standalone page (inside or outside of the gateway). This header can be empty (e.g. when called from outside of the gateway), can contain "Multiple" when displayed as a portlet in a page, or "Single" or "Hosted" when in an independent page accessed through the gateway.
- In the "Multiple" case, I do not return the 304 error but instead a Service Unavailable error (503). As per the setting above, the error is not displayed and the cached content is shown.
- In any other case, I return the 304 error, as a standalone page is properly processed by the browser. In such cases, the caching is done on the browser side and not the portal.

So far, so good. The performance has increased a lot, as could be expected, and the portlet server has more time to do other things than constantly rendering the same content...

Key lines of code:

Const c_sDateFormat As String = "yyyy-MM-dd HH:mm:ss"
Dim bFromPortal As Boolean = (Request.Headers("CSP-Aggregation-Mode") = "Multiple")

If dBrowserDate.ToString(c_sDateFormat) = dObjectDate.ToString(c_sDateFormat) Then
If bFromPortal Then
Response.StatusCode = System.Net.HttpStatusCode.ServiceUnavailable
Else
Response.StatusCode = System.Net.HttpStatusCode.NotModified
End If
Response.End()
End If

Response.Cache.SetCacheability(HttpCacheability.Public)
Response.Cache.SetRevalidation(HttpCacheRevalidation.AllCaches)
Response.Cache.SetLastModified(dObjectDate)

Wednesday, May 27, 2009

IE8 not recognized in ASP.Net 1.1 applications

As always, I install the latest versions of anything when they become available (sometimes even betas). So did I for Internet Explorer 8.0 when it was released... And all of a sudden, my trusty applications made with ASP.Net 1.1 went berzerk. Things that used to work with IE7 and Firefox simply stopped working. This included external components we bought, like the ComponentArt suite. NavBars could not be clicked, drop-down menus behaved in strange ways, and so on.

I dug a bit and found the problem. My browser was not detected as it should. The
"Request.Browser.Browser" command returned "Unknown" instead of the expected "IE". But why?

I remarked that on my Vista x64 machine, the 32 bits browser would have these issues, but not the 64 bits version of IE. I compared the "User-Agent" HTTP header and saw these values:

32 bits:
Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; WOW64; Trident/4.0; GTB6; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; InfoPath.2; .NET CLR 1.1.4322; .NET CLR 3.5.21022; .NET CLR 3.5.30729; .NET CLR 3.0.30618; FDM; OfficeLiveConnector.1.3; OfficeLivePatch.0.0)

64 bits:
Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC1; Media Center PC 5.0; .NET CLR 3.5.30729)

I checked the browscap.ini and machine.config files on the server, but couldn't change them in any useful way.

Could the difference in length be the problem? In a Microsoft article on TechNet (http://technet.microsoft.com/en-us/library/bb496341.aspx) they say that the length of this header should remain shorter than 200 characters. In the first case, it is definitely longer...

So I dug further, looking for ways to shorten that User-Agent string. Many articles and blog talked about the following key to change in the registry:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Internet Settings\User Agent\Post Platform

I did this, but only the User-Agent of the 64 bits instance of IE seemed to care. I searched through the registry for the "OfficeLivePatch" key I can only see in the 32 bits instance, and found it it the following key:
HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Internet Settings\5.0\User Agent\Post Platform

As you can see, there is "Wow6432Node" in there, which corresponds to 32 bits applications running on a 64 bits OS. Exactly my case ;-)

I renamed the "Post Platform" key into "Post Platform-" and restarted the browser and... bingo :-) The browser is detected as IE 8.0 and everything works fine...

Now I need to find out why, when I open a new tab or a new instance of IE 8, the content is not loaded, as it continuously shows "Connecting...". I need to open one or more tabs before a connection can be made. It also happens when opening popup windows, which is even more annoying (in that case, I need to reopen the popup with Ctrl-N until it works).

To be continued...

Friday, January 4, 2008

Access a WinDev/HyperFile DB from ASP.Net

I spent some time figuring this out, so this might be useful to someone else...

Mission: access a WinDev DB through ODBC from a C# Web Service

Steps:

This should be it ;-)

Thanks WinDev for making our work so complicated...