Search This Blog

Thursday, January 17, 2013

Export of Lotus Notes e-mail to eml-file

Hi

Recently I started a series of posts on my blog about different approaches for Lotus Notes data exporting.
My first post about exporting of richtext item content is here.

Today I want to publish a new post about different options for exporting a standard Lotus Notes e-mail to an eml-file that can be opened in MS Outlook, Mozilla Thunderbird, Apple Mail and other mail clients.

On the bottom of this post you will find a link to a Notes database with examples I prepared for you.
This database contains four Lotus Notes agents which mainly implement the same idea about exporting Lotus Notes e-mail but in a slightly different way. When I say "same idea", I mean an exporting of Lotus Notes e-mail to MIME HTML with a conversion of richtext item (Body) to MIME if it is required.


There are following Notes Agents in my database:
  1. Get eml-file using UI MIME-conversion + DXL
  2. Get eml-file using Backend MIME-conversion + DXL
  3. Get eml-file using UI MIME-conversion + manual exporting
  4. Get eml-file using Backend MIME-conversion + manual exporting
Brief details about these examples:

1) Get eml-file using UI MIME-conversion + DXL: this solution is made completely by me. I mean I did not use any solutions accessible in Internet in this case. 
The main idea of this implementation is that Lotus Notes document can be exported to dxl with a special option MIMEOPTION_DXL for NotesDXLExporter object. This option gives you almost ready for using MIME HTML. All you need to do is to convert back special HTML symbols which were converted to html code during exporting. I mean that dxl-file you got after using of NotesDXLExporter with option MIMEOPTION_DXL contains things like &amp;, &lt;,  &gt; and so on and you have to convert them back to &, <, > ... to make your MIME HTML ready for use in eml-file. 
Before exporting of Lotus Notes e-mail to dxl file we need to convert content of richtext item (Body) to MIME format. I do that using resave of target e-mail in UI using a form with a richtext field with "Store content as HTML and MIME" property enabled.

So the main limitation of this approach is that you can not use it in schedule agents.

2) Get eml-file using Backend MIME-conversion + DXL: this solution is the same as previous one except the way of conversion of Lotus Notes e-mail body to MIME format: with IBM Lotus Notes 8.5.1 and higher we have a new method for NotesDocument class - it is the NotesDocument.ConvertToMIME() method. This method is capable to convert a NotesDocument with all items to MIME so it makes our goal easier to achieve and allow us to use this solution in background agents.
Other things are the same as for Notes Agent №1.

3) Get eml-file using UI MIME-conversion + manual exporting: this solution I implemented using examples from http://www.openntf.org and http://www.nsftools.com. As you see from agent's name this implementation also uses UI-based approach for conversion of e-mail richtext body to MIME format but does not use option  with exporting Notes Document to dxl to get MIME HTML. This time we export e-mail headers and html-content of e-mail body "manually" using lotus script.

4) Get eml-file using Backend MIME-conversion + manual exporting: this solution uses NotesDocument.ConvertToMIME() method for conversion of e-mail body to MIME format and does exporting of e-mail headers and html-content of e-mail body "manually" using lotus script.

Note*: Some words about Sub OutlookHeadersFixing that I use in all Notes Agents. I made this function for correction of some headers in eml-file using values from Notes-email. For example, to display a sent date of exported Lotus Notes e-mail in MS Outlook I had to create a header field "SendOn" and fill it up with a value of "PostedDate" item from Lotus Notes e-mail. 

Note**: all agents in my databases use some UI classes for Demo purposes but only 1st and 3rd really need UI for a work. Agents №2 and №4 do not need UI objects so you may get rid from them and you have  to do that in case of using these solutions in background agents - I just remind you about that.


Note***: I hope this post can be helpful for you. I know that some or even all agents I've done as examples can have some defects or shortcomings but my goal was to help you to start your own development. I've made this post many years ago and at that time I was a bit different developer :-)
I can only confirm that it is possible to make a real production solution based on my input but it may take some time ti make it perfect. Unfortunatelly it is hard for me to find time to reply on all your questions, bugs descriptions or personal emails in connection with this post so please excuse me. However I can do commercial development for you, I take 25 EUR/h and usually 2 days should be enough to make a working prototype.

34 comments:

  1. Hi,

    How are you?

    Very nice study materials.

    Actually I am going to read .nsf file using C#. Everything is going fine no issue. But in case of SENT Mail reading, I am only getting text of the mail content instead of html of the mail content.
    I have searched a lot in internet to fix this issue, but I did not get any helps.
    So will you please help me, how can I read html of mail content in case of SENT mails.

    Thank you very much...

    Best Regards,

    Jignesh

    ReplyDelete
  2. This comment has been removed by the author.

    ReplyDelete
  3. Hello Jignesh

    1) I do not think that this is the issue.
    Lotus Notes email can be sent either in MIME format or in Notes Richtext format.
    It depends from Lotus Notes configuration.
    Check this screen to get what I mean:
    https://docs.google.com/file/d/0B7GvEvzLy0NLRV8yZHBySDlYeUE/edit

    2) You need to convert required emails to MIME format before doing your things. You have many options here:

    2.1 You may try to export email to DXL with option MIMEOPTION_DXL using NotesApi toolkit that can be downloaded from here:
    https://www.ibm.com/developerworks/lotus/documentation/capi/

    2.2 You may try to use next functions from nnotes.dll (it is located in IBM\Lotus\Notes\)

    Declare Function HTMLCreateConverter Lib "NNOTES" Alias "HTMLCreateConverter" ( HtmlHandle As Long) As Integer
    Declare Function HTMLDestroyConverter Lib "NNOTES" Alias "HTMLDestroyConverter" ( ByVal HtmlHandle As Long) As Integer
    Declare Function HTMLConvertNote Lib "NNOTES" Alias "HTMLConvertNote" ( ByVal HtmlHandle As Long, ByVal DbHandle As Long, ByVal NoteHandle As Long, ByVal UrlArgsCount As Long, ByVal NullUrlArgs As Long) As Integer
    Declare Function HTMLGetPropertyLong Lib "NNOTES" Alias "HTMLGetProperty" ( ByVal HtmlHandle As Long, ByVal PropertyType As Long, RetVal As Long) As Integer

    I have an examples of using these functions here: http://ypastov.blogspot.com/2013/01/compilation-of-richtext-item-conversion.html

    2.3 But I would do next thing. I would create a separate simple NotesDatabase with a simple NotesAgent that can convert email using native NotesDocument.ConvertToMime() function (it's a lotusscript).
    After that I would copy required outbound email to this database using C#, run this agent on copied email using C# and then do all other things you already do with converted email. I think it is the easiest way.

    ReplyDelete
  4. Hi,
    Thanks for your answer. I am going to use NotesApi toolkit.
    I am using nnotes.dll C API with c# code. But i get exception saying:

    "Additional information: Attempted to read or write protected memory. This is often an indication that other memory is corrupt."

    Do you have any idea about this?

    Thank you very much for your help..

    ReplyDelete
  5. Thank you for providing a good example program
    Implementation of the four agents will the following error message

    The "INITIALIZE / OUTLOOKHEADERSFIXING: Object variable not set in line 24"

    To do an amendment is not executed until the correct

    mailUnid = mailCpy.Universalid
    Delete mailCpy
    Set mailCpy = db.Getdocumentbyunid (mailUnid)

    Else
    The Set mailCpy = mail 'here
    End If


    Do not know such an amendment is in line with the original design concept

    ReplyDelete
    Replies
    1. Hi

      Thanks. I just realized what you talked about :-) The issue was with emails which already were MIME and did not require conversion.
      I just updated all agents in my database to fix this issue.

      Delete
  6. It can not convert emails larger then 200 MB.

    ReplyDelete
  7. it is not working with larger amount of data. I want to convert my Bulk mailbox not only emails to EML format. How can i do that? Do you have any idea? please tell me

    ReplyDelete
    Replies
    1. Hi, did you figure out a way to do this? I'm trying to do the same.

      I have found some useful perl scripts that access LotusScript functions through Win32::OLE, however they only dump the documents as plain text, not valid .EML files. Ideally I would like them to be identical to when you drag and drop an email out of Notes, but you can only do 50 at a time that way.

      Yuriy, the ypastov.eml.nsf download no longer seems to be accessible?

      Delete
    2. >>Yuriy, the ypastov.eml.nsf download no longer seems to be accessible?

      it is still accessible
      try Chrome or newer IE

      Delete
    3. I was using Chrome already... I tried again and it worked, no idea why! Thanks.

      I presume I need to install designer to look at and/or modify your code to handle multiple documents? When I try to use the actions they say "Cannot locate Form: MimeConvert".

      Delete
  8. To make agent nr.2 work (anf I guess the other as well) you need to modify the code. Otherwise MIME mails will fail (only rich text mails will work):
    Here is the modification:
    'First thing: let's check if current email's Body is stored as MIME
    'If not - then we need to convert Email's Body to MIME

    Set mailCpy = db.Createdocument()
    Call mail.Copyallitems(mailCpy, True)

    Set mime = mail.GetMIMEEntity

    If mime Is Nothing Then
    Call mailCpy.Replaceitemvalue("Form", "MimeConvert")
    Call mailCpy.Converttomime(2)
    End If

    ReplyDelete
  9. Thanks Andreas :-)
    I appreciate that you spent your time and debugged the issue.
    I just updated all agents with this fix.

    ReplyDelete
  10. You are welcome, Yuriy, thanks to you for the code. Very useful

    ReplyDelete
  11. Hi Youriy,

    may be a good ideao to remplace Lsi_info calls with GetThreadInfo call to avoid client crashes

    Msgbox Lsi_info(12) & "/" & Lsi_info(2) & ": " & Error & " in line " & Erl
    becomes
    Msgbox Getthreadinfo(10) & "/" & Getthreadinfo(1) & ": " & Error & " in line " & Erl

    see http://blog.gollmick.de/mgoblog.nsf/dx/Transforming_LSI_info_to_GetThreadInfo.htm

    ReplyDelete
  12. This comment has been removed by a blog administrator.

    ReplyDelete
  13. This comment has been removed by the author.

    ReplyDelete
  14. This comment has been removed by the author.

    ReplyDelete
  15. Hi
    DXL-file created with MIMEOPTION_DXL contains data we need for creation of eml-file but also contains other tags which a specific only for DXL.
    DXL is Domino XML, it has different format than eml-file has.
    So to get eml-file from DXL file I need to cut off DXL specific things from dxl-file and do small corrections.
    I use XSL transformation to get rid of XML specific things (if I remember right)

    ReplyDelete
  16. Hello,

    I can't se the code of the database. Is the Design hidden?

    Best regards

    Manuela

    ReplyDelete
  17. Yuri, I noticed the picklist does not allow you to select more than one document; it's because you have it set to "false" in the picklistcollection.

    also, if you'd like this to work OS independently, you should query the OS and change the temp path + direction of the \ too, something like:

    select case session.platform
    case "Windows/32"

    case "Linux"

    case else
    Print "Platform not supported"

    end case

    ReplyDelete
  18. Thanks, this saved me a lot of time!

    ReplyDelete
  19. Thank you for this, it's very useful.
    I can't get it to work if the email has an inline image, though.
    Do you have any ideas to fix this, please?

    ReplyDelete
  20. Hi Yuriy, this is great stuff. I am using your agent no. 2. I added the following to SpecialHTMLSymbolsFixing:
    ReplaceTo(5) = {X-Notes-Item: Memo; name=Form}
    ReplaceFrom(5) = {X-Notes-Item: MimeConvert; name=Form}
    in the appropriate places. So an exported Mail can be reopened by Notes without causing an error "Unknow Form: MimeConvert".

    ReplyDelete
  21. Where is the Source Code for this convertor?

    ReplyDelete
  22. Hi Yuriy,
    Very nice post :).
    I tried to write an Java application (from NSF to eml) on Linux (centos 6), but i got a weird exception on the method convertToMIME:
    Begin CD to MIME Conversion(Process: ? (00003CB5:00000002), Database: /home/sample.nsf, Note: 00000926)
    HTMLCreateConverter> File does not exist
    OCStartup> File does not exist
    CD to MIME error (Process: ? (00003CB5:00000002), Database: /home/sample.nsf, Note: 00000926): File does not exist
    This exception drove me crazy. Any idea for this kind of exception?
    Thanks

    ReplyDelete
  23. Юрий, спасибо за пример! Подскажите, если в полях нотес документа "SendTo", "CopyTo" имена лотус были вида "Вячеслав/Company" (на русском языке), то после конвертации в eml в соответствующем поле имеем что-то вида "0xL5C2FFF7E5F1EBE0E2z/Company". Можно это как то поправить в процессе преобразования, чтоб оставалось как исходное - "Вячеслав/Company" ?

    ReplyDelete
  24. Hi Yuriy,

    Great post. I was searching for help on converting eml to nsf. There are lot of posts talking about nsf to eml conversion , but not the other way. It would be great if you could provide insight on any of the available API that could convert eml to nsf.

    Thanks in advance
    Nuppu

    ReplyDelete
    Replies
    1. I can do such conversion and provide you with source code.
      25EUR/h.

      Delete
  25. Hello Yuriy , many thanks for your code, I use this to export mails from DMS database to filesystem and in the most cases that works fine. Sometimes I got an error message in function Printmime

    Call mime.GetEntityAsText(stream)

    PrintMime in Zeile: 13: Stream text line too long to handle (> 50K). I could not fin any information about limits. Did you habe the same error?

    If you have an idea? thanks in advance
    Lutz.Geschinsky@weilgut.de

    ReplyDelete
    Replies
    1. You might get around this problem by converting the content to Base64:
      Call mime.Encodecontent(1727)
      Call mime.Getentityastext(stream)

      Delete
  26. Yuriy, just wanted to let you know that this code is still helpful. :) Just pulled it down yesterday and performed some of the code updates that those above have recommended for resolving the "Unknown Form: MimeConvert" error. I also have it prompting me for a folder name so that I can get the emails from that folder and then dump them all into a folder with the same name as .eml files.

    Thanks again for this!

    ReplyDelete
  27. I have the exact same issue !
    Unfortunately the EncodeContent() function in "ENC_BASE64" (1727) is not helping.
    After investigations of the source mails, they were generated by Applications under Linux and provided HTML without any CR\LF (EOL), so the whole HTML data is on unique line and exceed the 50 KB limit !
    => How to split the huge line into multiple and smaller chunks for Stream writing.
    Thank you for your investigation and answer.
    olivier

    ReplyDelete
  28. Unfortunately the first two methods crash my HCL Note 11.0.1FP3 on Mac OS Big Sur 11.3. Last two give respectively "Illegal function call in line 85" and "line 82".

    ReplyDelete